Texture synthesis and transfer from multiple samples

Computer graphics applications often use textures to decorate virtual objects without modeling geometric details. These textures can be seamlessly generated from sample textures using texture synthesis algorithms. However, most existing texture synthesis algorithms are designed for rectangular domains and cannot be easily extended to general surfaces. In this paper, we present a novel method for texture synthesis and texture growing on triangle mesh with arbitrary topology. Using the sampled texture patches as the building blocks for texture synthesis, the visually seamless texture can be synthesized for a wide variety of textures ranging from regular to stochastic. The synthesizing process generates an initial texture block from the input texture patches and then iteratively searches for all texture blocks from the sampled texture patches to select a patch which optimizes the smoothness of the overlapping boundaries of the synthesized texture. It is showed that the proposed scheme is effective and efficient for synthesizing and growing of a wide range of textures on surfaces.

Texture synthesis is the hot research topic in the field of computer vision, computer graphics and image processing. Sample-based texture synthesis method has been proposed and becomes a new texture tiling technique with the development of the texture mapping and procedural texture synthesis. Image Quilting stitching algorithm is an ideal algorithm for the texture stitching. In this article, by using the image Quilting algorithm to finish the texture transfer and stitch the objective texture image, whose texture style does not exist in the initial image. In experiments, by transferring the style of the human face image and character image respectively, verified the effectiveness of this proposed algorithm.

Many natural and man-made surface patterns are created by interactions between texture elements and surface geometry. We believe that the best way to create such patterns is to synthesize a texture directly on the surface of the model. Given a texture sample in the form of an image, we create a similar texture over an irregular mesh hierarchy that has been placed on a given surface.Our method draws upon texture synthesis methods that use image pyramids, and we use a mesh hierarchy to serve in place of such pyramids. First, we create a hierarchy of points from low to high density over a given surface, and we connect these points to form a hierarchy of meshes. Next, the user specifies a vector field over the surface that indicates the orientation of the texture. The mesh vertices on the surface are then sorted in such a way that visiting the points in order will follow the vector field and will sweep across the surface from one end to the other. Each point is then visited in turn to determine its color. The color of a particular point is found by examining the color of neighboring points and finding the best match to a similar pixel neighborhood in the given texture sample. The color assignment is done in a coarse-to-fine manner using the mesh hierarchy. A texture created this way fits the surface naturally and seamlessly.

An improved image texture synthesis based on algorithm convolution neural network

In the research of virtual hepatic surgery, the synthesis of hepatic texture space and the texture mapping of the hepatic body model are inseparable. Different methods for hepatic volume texture on the authenticity and mapping speed tend to have much difference. Based on computer unified device architecture (CUDA), this paper tries to present a texture synthesis and mapping method for the accelerating of hepatic body, which is used to solve the problem that traditional method under the hepatic volumetric texture synthesis and mapping process cost too much time. This new method may eliminate the traditional serial correlation of texture synthesis. What’s more, it may improve the efficiency of texture synthesis when the multithreading parallel processing for synthesis in the hepatic volumetric texture space in selecting and distributing blocks based on CUDA architecture.

This study presents a real-time texture transfer method for artistic style transfer for video stream. We propose a parallel framework using a T-shaped kernel to enhance the computational performance. With regard to accelerated motion estimation, which is necessarily required for maintaining temporal coherence, we present a method using a downscaled motion field to successfully achieve high real-time performance for texture transfer of video stream. In addition, to enhance the artistic quality, we calculate the level of abstraction using visual saliency and integrate it with the texture transfer algorithm. Thus, our algorithm can stylize video with perceptual enhancements.

Texture synthesis is a fundamental problem in computer graphics that would benefit various applications. Existing methods are effective in handling 2D image textures. In contrast, many real-world textures contain meso-structure in the 3D geometry space, such as grass, leaves, and fabrics, which cannot be effectively modeled using only 2D image textures. We propose a novel texture synthesis method with Neural Radiance Fields (NeRF) to capture and synthesize textures from given multi-view images. In the proposed NeRF texture representation, a scene with fine geometric details is disentangled into the meso-structure textures and the underlying base shape. This allows textures with meso-structure to be effectively learned as latent features situated on the base shape, which are fed into a NeRF decoder trained simultaneously to represent the rich view-dependent appearance. Using this implicit representation, we can synthesize NeRF-based textures through patch matching of latent features. However, inconsistencies between the metrics of the reconstructed content space and the latent feature space may compromise the synthesis quality. To enhance matching performance, we further regularize the distribution of latent features by incorporating a clustering constraint. In addition to generating NeRF textures over a planar domain, our method can also synthesize NeRF textures over curved surfaces, which are practically useful. Experimental results and evaluations demonstrate the effectiveness of our approach.

Sample based texture synthesis allows the generation of arbitrarily sized textures based on a small sample image. One main drawback of the technique is that a new pixel or patch is selected only considering local information in the generated texture. Thus structural error can't assure to be avoided for structural textures. To accelerate texture synthesis process and to avoid structural mismatch in the output picture, we present in this paper a structural pattern analysis (SPA) algorithm and a texture synthesis method using the algorithm. The pattern analysis tool also quantizes the texture randomness in macro scope view. It can be used in other applications such as automatic texture classification and quality evaluation of synthesized texture. We illustrate our result with some examples.

Texture synthesis has been an active research area in computer graphics, vision, and image processing. A hybrid texture synthesis approach that combines the strength of Ashikhmin’s [1] and Liang’s [11] algorithms is presented in this paper. Using patches from the input sample texture to extend the input itself, more suitable valid candidate pixels in the input sample can be more precisely determined. An extra original position array is used to record the valid pixel position in the extended region of input sample texture such that the candidate pixel can be retrieved efficiently. Pasting patches with more global information of texture features to initialize the output texture, the texture synthesis process tends to grow larger pieces in the output texture so that the perceptual similarity is reserved as much as possible. Many classes of textures have been used to test our approach. As the experimental results showed, the proposed hybrid approach effectively and efficiently synthesizes high-quality textures from a wide variety of textures.

We present an efficient multiresolution block sampling-based texture synthesis algorithm that works well for a wide variety of challenging textures. We consider the texture synthesis as a procedure that first estimates and then resamples a probability function. To achieve the effect of good and fast estimation of the probability function for a given texture as well as to retain the structural information during synthesis, we propose a concept of block sampling and a corresponding novel scheme of texture synthesis based on multiresolution block sampling (MBS). The computational complexity of the proposed algorithm is much lower than that of existing algorithms. Experimental results have shown that his novel texture synthesis algorithm produces superior results for many challenging textures.

In this paper, the layer choices of the image style transfer method using the VGG-19 neural network are studied. The VGG-19 network is used to extract the feature maps which have their implicit meaning as a learning basis. If the layers for stylistic learning are not suitably chosen, the quality of style transferred image may not look good. After making experiments, it can be observed that the color information is concentrated on lower layers from conv1-1 to conv2-2, and texture information is concentrated on the middle layers from conv3-1 to conv4-4. As to the higher layers from conv5-1 to conv5-4, they seem to be able to depict image content well. Based on these observations, the methods of color transfer, texture transfer and style transfer are presented and make comparisons with conventional methods.

Direction fields and vector fields play an increasingly important role in computer graphics and geometry processing. The synthesis of directional fields on surfaces, or other spatial domains, is a fundamental step in numerous applications, such as mesh generation, deformation, texture mapping, and many more. The wide range of applications resulted in definitions for many types of directional fields: from vector and tensor fields, over line and cross fields, to frame and vector-set fields. Depending on the application at hand, researchers have used various notions of objectives and constraints to synthesize such fields. These notions are defined in terms of fairness, feature alignment, symmetry, or field topology, to mention just a few. To facilitate these objectives, various representations, discretizations, and optimization strategies have been developed. These choices come with varying strengths and weaknesses. This course provides a systematic overview of directional field synthesis for graphics applications, the challenges it poses, and the methods developed in recent years to address these challenges.

This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative neural networks for efficient texture synthesis. While deep neural network approaches have recently demonstrated remarkable results in terms of synthesis quality, they still come at considerable computational costs (minutes of run-time for low-res images). Our paper addresses this efficiency issue. Instead of a numerical deconvolution in previous work, we precompute a feed-forward, strided convolutional network that captures the feature statistics of Markovian patches and is able to directly generate outputs of arbitrary dimensions. Such network can directly decode brown noise to realistic texture, or photos to artistic paintings. With adversarial training, we obtain quality comparable to recent neural texture synthesis methods. As no optimization is required any longer at generation time, our run-time performance (0.25M pixel images at 25Hz) surpasses previous neural texture synthesizers by a significant margin (at least 500 times faster). We apply this idea to texture synthesis, style transfer, and video stylization.

Seamless montage for texturing 3D models is very important in 3D video games and computer animations. The goal is to generate an arbitrarily large texture from a small sample image while both the sample texture and the result texture must be perceived by human observers to be the same and there is no obvious seam. This is slightly different from texture synthesis whose main principle is that the result texture must be non-periodical. Since there exist substantial periodical textures in real world, this paper presents a method for seamless montage without seams to form periodical textures. First we apply a quilting algorithm together with Wang Tiles to design the tile for montage, and we make comparison between them in terms of practical maneuverability. Then we employ the tile designed through Wang Tiles for seamless montage to generate a large texture with an arbitrary size. We also consider the montage of 3D surface textures under various illumination directions.

We propose a layered image inpainting scheme based on image decomposition. The damaged image is first decomposed into three layers: cartoon, edge, and texture. The cartoon and edge layers are repaired using an adaptive offset operator that can fill-in damaged image blocks while preserving sharpness of edges. The missing information in the texture layer is generated with a texture synthesis method. By using discrete cosine transform (DCT) in image decomposition and trading between resolution and computation complexity in texture synthesis, the processing time is kept at a reasonable level.