Comic style transfer based on generative confrontation network

This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches based on deep neural networks typically use second-order statistics to generate the output. However, these hand-crafted features computed from a single image cannot leverage style information sufficiently, which leads to artifacts such as local distortions and style inconsistency. To address these issues, we learn style representation directly from a large number of images based on contrastive learning by considering the relationships between specific styles and the holistic style distribution. Specifically, we present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components: a parallel contrastive learning scheme for style representation and transfer, a domain enhancement (DE) module for effective learning of style distribution, and a generative network for style transfer. Qualitative and quantitative evaluations show the results of our approach are superior to those obtained via state-of-the-art methods. The code is available at https://github.com/zyxElsa/CAST_pytorch .

Stylized image denoising via noise style transfer and Quasi Siamese network

The rapid development of artificial intelligence technology and deep learning technology has also promoted the popularization and application of image style transfer in all walks of life. In the traditional ceramic industry, the application of image style transfer technology is still rare. This article is based on the style transfer training of ceramic blue and white images and uses three methods of rapid style transfer, direct style transfer and image pre-processing style transfer to conduct experiments. According to the results of image quality evaluation, it is concluded that the method suitable for the style transfer of ceramic blue and white images is the image pre-processing style transfer method.

To improve the expressiveness and realism of illustration images, the experiment innovatively combines the attention mechanism with the cycle consistency adversarial network and proposes an efficient style transfer method for illustration images. The model comprehensively utilizes the image restoration and style transfer capabilities of the attention mechanism and the cycle consistency adversarial network, and introduces an improved attention module, which can adaptively highlight the key visual elements in the illustration, thereby maintaining artistic integrity during the style transfer process. Through a series of quantitative and qualitative experiments, high-quality style transfer is achieved, especially while retaining the original features of the illustration. The results show that when running on the Monet2photo dataset, when the system iterates to 72 times, the loss function value of the research method approaches the target value of 0.00. On the Horse2zebra dataset, as the sample size increases, the research method has the smallest FID value, and the value approaches 40.00 infinitely. With the change of peak signal-to-noise ratio, the accuracy of the research algorithm has been greater than 95.00%. Practical application found that the color of the image obtained by the research method is more gorgeous and the line features are more obvious. The above results all show that the research method has achieved more satisfactory results in the task of style transfer of illustration images, especially in terms of the accuracy of style transfer and the retention of image details.

The paper proposes a solution to effectively handle salient regions for style transfer between unpaired datasets. Recently, Generative Adversarial Networks (GAN) have demonstrated their potentials of translating images from source domain X to target domain Y in the absence of paired examples. However, such a translation cannot guarantee to generate high perceptual quality results. Existing style transfer methods work well with relatively uniform content, they often fail to capture geometric or structural patterns that always belong to salient regions. Detail losses in structured regions and undesired artifacts in smooth regions are unavoidable even if each individual region is correctly transferred into the target style. In this paper, we propose SDP-GAN, a GAN-based network for solving such problems while generating enjoyable style transfer results. We introduce a saliency network, which is trained with the generator simultaneously. The saliency network has two functions: (1) providing constraints for content loss to increase punishment for salient regions, and (2) supplying saliency features to generator to produce coherent results. Moreover, two novel losses are proposed to optimize the generator and saliency networks. The proposed method preserves the details on important salient regions and improves the total image perceptual quality. Qualitative and quantitative comparisons against several leading prior methods demonstrates the superiority of our method.

Image style transfer is a kind of image-to-image conversion method, which has been widely used in the field of computer vision and is an important research direction. In the application of target deformation, the traditional algorithm has the disadvantages of unclear positioning and difficult separation from the background. In this paper, an improved cyclic generative antagonistic network (CycleGAN) is proposed. The model adds a layer of self-attention algorithm to the discriminator structure, and uses clues from all feature locations to establish global dependence, to obtain global geometric features of images, and to achieve long correlation. Compared with CycleGAN algorithm, the results show that the algorithm in this paper can learn to generate a style transfer image with clear texture and full rendering, and at the same time the image has no obvious loss in details. Compared with the original algorithm, the convergence speed in the learning process of the algorithm in this paper is obviously improved, the oscillation amplitude of loss function is small, and the number of steps required for network convergence is reduced. In the process of testing, the algorithm stylizes 1163 images, and the consumption time of each generation only increases by 17 s.

In this work, we focus on building style transfer, which transforms ruin or old buildings to modern architecture. Inspired by Gaty's and Goodfellow's style transfer and generative adversarial network (GAN), we use CycleGAN to conquer this type of problem. As we know, image style transfer usually generated unexpected artifacts. To avoid the artifacts and generate better images, we add so called “perception loss” into the network, which is the feature loss extracted by VGG pre-trained model. In the part of “cycle” structure, we adjust cycle loss by changing the ratio of weighting parameters. Finally, we collect images of both ruin (or old) and modern architecture from websites and use unsupervised learning to train the model. The experimental results show our proposed method indeed realize the modern architecture style transfer for ruin or old buildings.

To provide semantic image style transfer results which are consistent with human perception, transferring styles of semantic regions of the style image to their corresponding semantic regions of the content image is necessary. However, when the object categories between the content and style images are not the same, it is difficult to match semantic regions between two images for semantic image style transfer. To solve the semantic matching problem and guide the semantic image style transfer based on matched regions, we propose a novel semantic context-aware image style transfer method by performing semantic context matching followed by a hierarchical local-to-global network architecture. The semantic context matching aims to obtain the corresponding regions between the content and style images by using context correlations of different object categories. Based on the matching results, we retrieve semantic context pairs where each pair is composed of two semantically matched regions from the content and style images. To achieve semantic context-aware style transfer, a hierarchical local-to-global network architecture, which contains two sub-networks including the local context network and the global context network, is proposed. The former focuses on style transfer for each semantic context pair from the style image to the content image, and generates a local style transfer image storing the detailed style feature representations for corresponding semantic regions. The latter aims to derive the stylized image by considering the content, the style, and the intermediate local style transfer images, so that inconsistency between different corresponding semantic regions can be addressed and solved. The experimental results show that the stylized results using our method are more consistent with human perception compared with the state-of-the-art methods.

With the rapid development of computer computing power, as an important method in the field of artificial intelligence, deep learning has amazing learning ability, especially in dealing with massive data, which makes deep learning in the fields of image recognition, image classification, natural language processing, data mining and unmanned driving, Has shown an extraordinary role. In previous studies, the style transfer algorithm has not developed well due to the poor computing power of Computer, the basic configuration of computer hardware can not meet the minimum requirements and the poor image effect after migration. However, with the development of computer hardware and the rapid change of GPU computing power, the style transfer network based on deep learning has become a hot issue in the study of style transfer in recent years. According to the research, although the traditional style transfer method can obtain the texture, color and other information of the style image, the model needs to be learned every time a new target image is generated, and the time cost during this period is very high. In this way, the trained model is not repeatable, and the generated image is often very random and can not get good results. Therefore, the emergence of style transfer methods based on deep learning solves the limitations of traditional style transfer methods. Style transfer methods based on deep learning are faster than traditional style transfer methods, and the generalization of the model is better. The style transfer algorithms of main neural networks are divided into two categories, Slow style transfer based on image iteration and fast style transfer based on model iteration. VGG network model can combine style image and content image, and greatly improve the style transfer efficiency of image.

Style transfer means that the characteristic information of the style image is transferred to the content image under a given content and style picture. Meanwhile, the transferred image is faithful to the content image. Currently, the transferred image has many problems such as artifacts and distortion of the spatial structure. To solve these problems, a dynamic residual-multi-level and multi-pooling network combined with our improved style transfer algorithm is proposed in this paper to achieve better effect of arbitrary image style transfer. To elaborate more specifically, First of all, We add a dynamic-residual network layer to the network to speed up the training speed of the network model and improve the robustness of the network. Secondly, we use a multi-level and multi-pooling layer to extract more specific spatial structure, edge texture and other information in the image, and the network layer also has an additional denoising effect. Third, the improved style transfer algorithm aligns the mean and variance of the content and style images from a statistical point of view, and realizes adaptive arbitrary style transfer of the original input image features. Finally, during model training, the convergence speed of the proposed approach is faster than other current advanced methods, and the transferred renderings are better than other network models in quantitative comparisons such as SSIM and Gram. It is worth mentioning that our model has the fastest real-time transfer speed and can realize any image style transfer.

Image style transfer has become a key technique in modern photo-editing applications. Although significant progress has been made to blend content from one image with style from another image, the synthesized image may have a hallucinatory effect when the texture from the style image is rich when processing high-resolution image style transfer tasks. In this paper, we propose a novel attention mechanism, named compositional attention, to design a compositional transformer-based autoencoder (CTA) to solve this above-mentioned issue. With the support from this module, our model is capable of generating high-quality images when transferring from texture-riched style images to content images with semantics. Additionally, we embed region-based consistency terms in our loss function for ensuring internal structure semantic preservation in our synthesized image. Moreover, information theory-based CTA is discussed and Kullback–Leibler divergence loss is introduced to preserve more brightness information for photo-realistic style transfer. Extensive experimental results based on three benchmark datasets, namely Churches, Flickr Landscapes, and Flickr Faces HQ, confirmed excellent performance when compared to several state-of-the-art methods. Based on a user study assessment, the majority number of users, ranging from 61% to 66%, gave high scores on the transfer effects of our method compared to 9% users who supported the second best method. Further, for the questions of realism and style transfer quality, we achieved the best score, i.e., an average of 4.5 out of 5 compared to other style transfer methods.

Artistic text style transfer is the task of migrating the style from a source image to the target text to create artistic typography. Recent style transfer methods have considered texture control to enhance usability. However, controlling the stylistic degree in terms of shape deformation remains an important open challenge. In this paper, we present the first text style transfer network that allows for real-time control of the crucial stylistic degree of the glyph through an adjustable parameter. Our key contribution is a novel bidirectional shape matching framework to establish an effective glyph-style mapping at various deformation levels without paired ground truth. Based on this idea, we propose a scale-controllable module to empower a single network to continuously characterize the multi-scale shape features of the style image and transfer these features to the target text. The proposed method demonstrates its superiority over previous state-of-the-arts in generating diverse, controllable and high-quality stylized text.

At present, neural style transfer technique is gaining more and more popularity in different fields like sports and entertainment. During the artistic style transfer of an image, style loss and content loss occurs. Several research works are being performed in present with a view to reduce the rate of both style and content loss, but some of these processes require much time for style transfer. In this paper, we propose a style transfer method using convolutional neural network that minimizes the rate of style and content loss in a minimal time compared to some other works. The proposed method is applied on different images and artworks. The results are compared with some recent research works related to this method and the proposed method is found to be 5–7% more efficient and faster than those related works.

Deep learning has enabled image style transfer to make great strides forward. However, unlike many other styles, transferring the watercolor style to portraits is significantly challenging in image synthesis and style transfer. Pixel-correlation-based methods do not produce satisfactory watercolors. This is because portrait watercolors exhibit the sophisticated fusion of various painting techniques in local areas, which poses a problem for convolutional neural networks to accurately handle fine-grained features. Moreover, the common but problematic way of coping with multiple scales greatly impedes the performance of existing style transfer methods with fixed receptive fields. Although it is possible to develop an image processing pipeline mimicking various watercolor effects, such algorithms are slow and fragile, especially for inputs of different scales. As a remedy, this paper proposes WCGAN, a generative adversarial network (GAN) architecture dedicated to watercolorization of portraits. Specifically, a novel localized style loss suitable for watercolorization is proposed to deal with local details. To handle portraits of different scales and improve robustness, a novel discriminator architecture with three parallel branches of varying sizes of receptive fields is introduced. In addition, the application of WCGAN is expanded to video style transfer where a novel kind of video training data based on random crops is developed to efficiently capture temporal consistency. Extensive experimental results from qualitative and quantitative analyses demonstrate that WCGAN generates state-of-the-art, high quality watercolors from portraits.

With the rapid development of deep learning technology, style transfer has achieved significant breakthroughs at both algorithmic and application levels, greatly promoting innovative interactions between content and style. This paper comprehensively reviews the fundamental concepts, classifications, and widespread applications of style transfer within neural networks, with a particular focus on the principles, diverse variants, and synthesis techniques of neural network-based style transfer. Additionally, the article explores image style transfer as a research hotspot in the field of artificial intelligence and computer vision, highlighting that traditional methods primarily rely on physical and texture synthesis techniques, which often result in coarse outcomes and poor robustness. Through comparative analysis, this paper outlines the development trajectory and latest research advances in image style transfer and proposes future research directions.