Illustration image style transfer method design based on improved cyclic consistent adversarial network.

Stylized image denoising via noise style transfer and Quasi Siamese network

With the rapid development of computer computing power, as an important method in the field of artificial intelligence, deep learning has amazing learning ability, especially in dealing with massive data, which makes deep learning in the fields of image recognition, image classification, natural language processing, data mining and unmanned driving, Has shown an extraordinary role. In previous studies, the style transfer algorithm has not developed well due to the poor computing power of Computer, the basic configuration of computer hardware can not meet the minimum requirements and the poor image effect after migration. However, with the development of computer hardware and the rapid change of GPU computing power, the style transfer network based on deep learning has become a hot issue in the study of style transfer in recent years. According to the research, although the traditional style transfer method can obtain the texture, color and other information of the style image, the model needs to be learned every time a new target image is generated, and the time cost during this period is very high. In this way, the trained model is not repeatable, and the generated image is often very random and can not get good results. Therefore, the emergence of style transfer methods based on deep learning solves the limitations of traditional style transfer methods. Style transfer methods based on deep learning are faster than traditional style transfer methods, and the generalization of the model is better. The style transfer algorithms of main neural networks are divided into two categories, Slow style transfer based on image iteration and fast style transfer based on model iteration. VGG network model can combine style image and content image, and greatly improve the style transfer efficiency of image.

This work presents Unified Contrastive Arbitrary Style Transfer (UCAST), a novel style representation learning and transfer framework, that can fit in most existing arbitrary image style transfer models, such as CNN-based, ViT-based, and flow-based methods. As the key component in image style transfer tasks, a suitable style representation is essential to achieve satisfactory results. Existing approaches based on deep neural networks typically use second-order statistics to generate the output. However, these hand-crafted features computed from a single image cannot leverage style information sufficiently, which leads to artifacts such as local distortions and style inconsistency. To address these issues, we learn style representation directly from a large number of images based on contrastive learning by considering the relationships between specific styles and the holistic style distribution. Specifically, we present an adaptive contrastive learning scheme for style transfer by introducing an input-dependent temperature. Our framework consists of three key components: a parallel contrastive learning scheme for style representation and transfer, a domain enhancement (DE) module for effective learning of style distribution, and a generative network for style transfer. Qualitative and quantitative evaluations show the results of our approach are superior to those obtained via state-of-the-art methods. The code is available at https://github.com/zyxElsa/CAST_pytorch .

Image style transfer has become a key technique in modern photo-editing applications. Although significant progress has been made to blend content from one image with style from another image, the synthesized image may have a hallucinatory effect when the texture from the style image is rich when processing high-resolution image style transfer tasks. In this paper, we propose a novel attention mechanism, named compositional attention, to design a compositional transformer-based autoencoder (CTA) to solve this above-mentioned issue. With the support from this module, our model is capable of generating high-quality images when transferring from texture-riched style images to content images with semantics. Additionally, we embed region-based consistency terms in our loss function for ensuring internal structure semantic preservation in our synthesized image. Moreover, information theory-based CTA is discussed and Kullback–Leibler divergence loss is introduced to preserve more brightness information for photo-realistic style transfer. Extensive experimental results based on three benchmark datasets, namely Churches, Flickr Landscapes, and Flickr Faces HQ, confirmed excellent performance when compared to several state-of-the-art methods. Based on a user study assessment, the majority number of users, ranging from 61% to 66%, gave high scores on the transfer effects of our method compared to 9% users who supported the second best method. Further, for the questions of realism and style transfer quality, we achieved the best score, i.e., an average of 4.5 out of 5 compared to other style transfer methods.

The rapid development of artificial intelligence technology and deep learning technology has also promoted the popularization and application of image style transfer in all walks of life. In the traditional ceramic industry, the application of image style transfer technology is still rare. This article is based on the style transfer training of ceramic blue and white images and uses three methods of rapid style transfer, direct style transfer and image pre-processing style transfer to conduct experiments. According to the results of image quality evaluation, it is concluded that the method suitable for the style transfer of ceramic blue and white images is the image pre-processing style transfer method.

Seismic data are widely used in oil, gas, and other kinds of mineral exploration and development. However, due to low artificial interpretation accuracy and small sample sizes, seismic data may not meet the needs of convolutional neural network training. There are major differences between optical image and seismic data, making it difficult for a model to learn seismic data characteristics. Therefore, a style transfer network is necessary to make the styles of optical image and seismic data more similar. Since the stylization effect of a seismic section is similar to that of most art styles, based on an in-depth study of image style transfer, this paper compared the effects of various style transfer models, and selected a Laplacian pyramid network to carry out a study of seismic section stylization. It transmits low-resolution global style patterns through a drafting network, revises high-resolution local details through correction networks, and aggregates all pyramid layers to output final stylized images of seismic profiles. Experiments show that this method can effectively convey the whole style pattern without losing the original image content. This style transfer method, based on the Laplacian pyramid network, provides theoretical guidance for the fast and objective application of the model to seismic data features.

We introduce a simple method for text style transfer that frames style transfer as denoising: we synthesize a noisy corpus and treat the source style as a noisy version of the target style. To control for aspects such as preserving meaning while modifying style, we propose a reranking approach in the data synthesis phase. We evaluate our method on three novel style transfer tasks: transferring between British and American varieties, text genres (formal vs. casual), and lyrics from different musical genres. By measuring style transfer quality, meaning preservation, and the fluency of generated outputs, we demonstrate that our method is able both to produce high-quality output while maintaining the flexibility to suggest syntactically rich stylistic edits.

Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their inputs. However, many existing systems purportedly designed for style transfer inherently warp the input's meaning through attribute transfer, which changes semantic properties such as sentiment. In this paper, we reformulate unsupervised style transfer as a paraphrase generation problem, and present a simple methodology based on fine-tuning pretrained language models on automatically generated paraphrase data. Despite its simplicity, our method significantly outperforms state-of-the-art style transfer systems on both human and automatic evaluations. We also survey 23 style transfer papers and discover that existing automatic metrics can be easily gamed and propose fixed variants. Finally, we pivot to a more real-world style transfer setting by collecting a large dataset of 15M sentences in 11 diverse styles, which we use for an in-depth analysis of our system.

Style transfer has been well studied in recent years with excellent performance processed. While existing methods usually choose CNNs as the powerful tool to accomplish superb stylization, less attention was paid to the latent style space. Rare exploration of underlying dimensions results in the poor style controllability and the limited practical application. In this work, we rethink the internal meaning of style features, further proposing a novel unsupervised algorithm for style discovery and achieving personalized manip-ulation. In particular, we take a closer look into the mechanism of style transfer and obtain different artistic style components from the latent space consisting of different style features. Then fresh styles can be generated by linear combination according to various style components. Experimental results have shown that our approach is superb in 1) restylizing the original output with the diverse artistic styles discovered from the latent space while keeping the content unchanged, and 2) being generic and compatible for various style transfer methods. Our code is available in this page: https://github.com/Shelsin/ArtIns.

Deep neural networks are vulnerable to adversarial examples crafted by applying human-imperceptible perturbations on clean inputs. Although many attack methods can achieve high success rates in the white-box setting, they also exhibit weak transferability in the black-box setting. Recently, various methods have been proposed to improve adversarial transferability, in which the input transformation is one of the most effective methods. In this work, we notice that existing input transformation-based works mainly adopt the transformed data in the same domain for augmentation. Inspired by domain generalization, we aim to further improve the transferability using the data augmented from different domains. Specifically, a style transfer network can alter the distribution of low-level visual features in an image while preserving semantic content for humans. Hence, we propose a novel attack method named Style Transfer Method (STM) that utilizes a proposed arbitrary style transfer network to transform the images into different domains. To avoid inconsistent semantic information of stylized images for the classification network, we fine-tune the style transfer network and mix up the generated images added by random noise with the original images to maintain semantic consistency and boost input diversity. Extensive experimental results on the ImageNet-compatible dataset show that our proposed method can significantly improve the adversarial transferability on either normally trained models or adversarially trained models than state-of-the-art input transformation-based attacks. Code is available at: https://github.com/Zhijin-Ge/STM.

Style transfer has received much attention, and Chinese character style transfer is an important application. In the previous Chinese character style transfer methods, most of the methods separately establish a mapping from source font to target Chinese character for each specific target font. It is impossible to extend a single model to other styles. The model training is based on the amount of Chinese character data from a single font, and the demand of the character is greater. In fact, different font style transfer tasks have certain commonalities. Therefore, we use multiple fonts in the data set and combine these fonts into multiple sets of Chinese character style transfer tasks. These tasks will be trained at the same time, instead of that only one font style transfer task training is achieved at a time like the previous work. In order to achieve the purpose of extracting and fusing the content features and style feature, we propose a style transfer network composed of content encoder, style encoder, feature fusion module, and decoder, based on the fusion of multi-scale content and style features. Through the experimental results, it can be found that, using our method, training the style transfer of multiple fonts at the same time can help any one of them to generate better results with only a few samples.

Artistic text style transfer is the task of migrating the style from a source image to the target text to create artistic typography. Recent style transfer methods have considered texture control to enhance usability. However, controlling the stylistic degree in terms of shape deformation remains an important open challenge. In this paper, we present the first text style transfer network that allows for real-time control of the crucial stylistic degree of the glyph through an adjustable parameter. Our key contribution is a novel bidirectional shape matching framework to establish an effective glyph-style mapping at various deformation levels without paired ground truth. Based on this idea, we propose a scale-controllable module to empower a single network to continuously characterize the multi-scale shape features of the style image and transfer these features to the target text. The proposed method demonstrates its superiority over previous state-of-the-arts in generating diverse, controllable and high-quality stylized text.

Style transfer refers to extracting styles from a specified style image template, and mapping the extracted style features to the content image without destroying the image content. At present, the use of deep learning methods for style transfer is one of the hotspots in the field of image research. This paper implements an image style transfer architecture based on cyclic consistent generative adversarial network, i. e., cyclic consistent confrontation network, and uses densely connected convolutional network to deepen the number of network layers. Test results show that stylized image with the proposed method is more obvious than that of the original cyclic consistent generative adversarial network, and the image quality is improved apparently.

To provide semantic image style transfer results which are consistent with human perception, transferring styles of semantic regions of the style image to their corresponding semantic regions of the content image is necessary. However, when the object categories between the content and style images are not the same, it is difficult to match semantic regions between two images for semantic image style transfer. To solve the semantic matching problem and guide the semantic image style transfer based on matched regions, we propose a novel semantic context-aware image style transfer method by performing semantic context matching followed by a hierarchical local-to-global network architecture. The semantic context matching aims to obtain the corresponding regions between the content and style images by using context correlations of different object categories. Based on the matching results, we retrieve semantic context pairs where each pair is composed of two semantically matched regions from the content and style images. To achieve semantic context-aware style transfer, a hierarchical local-to-global network architecture, which contains two sub-networks including the local context network and the global context network, is proposed. The former focuses on style transfer for each semantic context pair from the style image to the content image, and generates a local style transfer image storing the detailed style feature representations for corresponding semantic regions. The latter aims to derive the stylized image by considering the content, the style, and the intermediate local style transfer images, so that inconsistency between different corresponding semantic regions can be addressed and solved. The experimental results show that the stylized results using our method are more consistent with human perception compared with the state-of-the-art methods.

Caricature is an exaggerated pictorial representation of a person, which is widely used in entertainment and political media. Recently, GAN-based methods achieved automatic caricature generation through transferring caricature style and performing shape exaggeration simultaneously. However, the caricature synthesized by these methods cannot perfectly reflect the characteristics of the subject, whose shape exaggeration are not reasonable and requires facial landmarks of caricature. In addition, the existing methods always produce the bad cases in caricature style due to the simpleness of their style transfer method. In this paper, we propose a Style Attention based Global-local Aware GAN to apply the characteristics of a subject to generate personalized caricature. To integrate the facial characteristics of a subject, we introduce a landmark-based warp controller for personalized shape exaggeration, which employs the facial landmarks as control points to warp image according to its facial features, without requirement of the facial landmarks of caricature. To fuse the facial feature with caricature style appropriately, we introduce a style-attention module, which adopts an attention mechanism, instead of the simple Adaptive Instance Normalization (AdaIN) for style transfer. To reduce the bad cases and increase the quality of generated caricatures, we propose a multi-scale discriminator to both globally and locally discriminate the synthesized and real caricature, which improves the whole structure and realistic details of the synthesized caricature. Experimental results on two publicly available datasets, the WebCaricature and the CaVINet datasets, validate the effectiveness of our proposed method and suggest that our proposed method achieves better performance than the existing methods. The caricatures generated by the proposed method can not only preserve the identity of input photo but also the characteristic shape exaggeration for each person, which are highly close to the real caricatures drawn by real artists. It indicates that our method can be adopted in the real application.