Robust tile-based texture synthesis using texture element

We present an efficient multiresolution block sampling-based texture synthesis algorithm that works well for a wide variety of challenging textures. We consider the texture synthesis as a procedure that first estimates and then resamples a probability function. To achieve the effect of good and fast estimation of the probability function for a given texture as well as to retain the structural information during synthesis, we propose a concept of block sampling and a corresponding novel scheme of texture synthesis based on multiresolution block sampling (MBS). The computational complexity of the proposed algorithm is much lower than that of existing algorithms. Experimental results have shown that his novel texture synthesis algorithm produces superior results for many challenging textures.

This paper proposes Markovian Generative Adversarial Networks (MGANs), a method for training generative neural networks for efficient texture synthesis. While deep neural network approaches have recently demonstrated remarkable results in terms of synthesis quality, they still come at considerable computational costs (minutes of run-time for low-res images). Our paper addresses this efficiency issue. Instead of a numerical deconvolution in previous work, we precompute a feed-forward, strided convolutional network that captures the feature statistics of Markovian patches and is able to directly generate outputs of arbitrary dimensions. Such network can directly decode brown noise to realistic texture, or photos to artistic paintings. With adversarial training, we obtain quality comparable to recent neural texture synthesis methods. As no optimization is required any longer at generation time, our run-time performance (0.25M pixel images at 25Hz) surpasses previous neural texture synthesizers by a significant margin (at least 500 times faster). We apply this idea to texture synthesis, style transfer, and video stylization.

Many natural and man-made surface patterns are created by interactions between texture elements and surface geometry. We believe that the best way to create such patterns is to synthesize a texture directly on the surface of the model. Given a texture sample in the form of an image, we create a similar texture over an irregular mesh hierarchy that has been placed on a given surface.Our method draws upon texture synthesis methods that use image pyramids, and we use a mesh hierarchy to serve in place of such pyramids. First, we create a hierarchy of points from low to high density over a given surface, and we connect these points to form a hierarchy of meshes. Next, the user specifies a vector field over the surface that indicates the orientation of the texture. The mesh vertices on the surface are then sorted in such a way that visiting the points in order will follow the vector field and will sweep across the surface from one end to the other. Each point is then visited in turn to determine its color. The color of a particular point is found by examining the color of neighboring points and finding the best match to a similar pixel neighborhood in the given texture sample. The color assignment is done in a coarse-to-fine manner using the mesh hierarchy. A texture created this way fits the surface naturally and seamlessly.

In neighborhood-based texture synthesis, adjacent local regions need to satisfy color continuity constraints in order to avoid visible seams. Such continuity constraints seriously restrict the variability of synthesized textures, making it impossible to generate new textures by mixing multiple input textures with very different base colors. In this paper, we propose to relax such restrictions and decompose synthesis into two relatively disjoint stages. In the first stage, an intermediate synthesized texture is generated by only considering the high frequency details during region search and matching. Such a scheme broadens the search space during texture synthesis, but may produce obvious seams due to large discontinuities in low frequency components. In the second stage, instead of performing local feathering along these discontinuities, we perform Laplacian texture reconstruction, which retains the high frequency details but computes new consistent low frequency components to eliminate the seams. It does not only affect texels close to the discontinuities, but also modifies the rest of the texels. Therefore, it can be viewed as a global feature-preserving smoothing step, and is more effective than local feathering. Experiments indicate that our twostage synthesis can produce desirable results for regular texture synthesis as well as texture mixing from multiple sources.

We present an algorithm for synthesizing textures from an input sample. This patch-based sampling algorithm is fast and it makes high-quality texture synthesis a real-time process. For generating textures of the same size and comparable quality, patch-based sampling is orders of magnitude faster than existing algorithms. The patch-based sampling algorithm works well for a wide variety of textures ranging from regular to stochastic. By sampling patches according to a nonparametric estimation of the local conditional MRF density function, we avoid mismatching features across patch boundaries. We also experimented with documented cases for which pixel-based nonparametric sampling algorithms cease to be effective but our algorithm continues to work well.

Despite the rapid evolution of multimedia content, from 2D to 3D and to stereo on IMAX display, material and texture remain an indispensable component when rendering realistic and appealing graphics and animations. As the demand for high-definition displays increases, so is the necessity for high-resolution textures. Nevertheless, the available texture images very often have low resolution and are inadequate for high-quality rendering. Texture synthesis from examples offers an effective way not only for the creation of high resolution texture, but also useful in interpolating missing data resulted from unreliable transmission and overly compressed data. We present a new memory-efficient technique that facilitates high-quality texture synthesis. We compare the time performance and memory usage between our approach and the commonly used caching techniques. Experimental results show that our method can perform better, given limited memory resources.

Texture synthesis has been an active research area in computer graphics, vision, and image processing. A hybrid texture synthesis approach that combines the strength of Ashikhmin’s [1] and Liang’s [11] algorithms is presented in this paper. Using patches from the input sample texture to extend the input itself, more suitable valid candidate pixels in the input sample can be more precisely determined. An extra original position array is used to record the valid pixel position in the extended region of input sample texture such that the candidate pixel can be retrieved efficiently. Pasting patches with more global information of texture features to initialize the output texture, the texture synthesis process tends to grow larger pieces in the output texture so that the perceptual similarity is reserved as much as possible. Many classes of textures have been used to test our approach. As the experimental results showed, the proposed hybrid approach effectively and efficiently synthesizes high-quality textures from a wide variety of textures.

This paper proposes a new and efficient algorithm for block matching motion estimation. It reduces search times and improves performance of motion estimation. Instead of using the fixed first search point to be alike previous search algorithms to search motion vector, the proposed method finds more accurate first search point as to compensating search area using temporal correlation of motion vector. Therefore, the proposed method could reduce search times and improve the accuracy of motion estimation. The main idea of proposed method is based on the property of consistent directivity and center-biased distributions of motion vector. Simulation results show that the PSNR values are improved up to the 3.4 dB as depend on the image sequences and advanced about 1.5 dB on an average. The results of the comparison show that the performance of the proposed algorithm is better than those of other fast search algorithms whether the image sequence contains fast or slow motion, and is similar to the performance of the Full Search algorithm. Simulation results also show that the performance of the proposed scheme gives better subjective picture quality than the other fast search algorithms.

In this paper, we developed an auto-focus image quality evaluation function from the histogram feature function (HFF) of a high-speed camera. Based on this function, we then proposed a fast auto-focus search algorithm. We further demonstrated that the auto-focus image quality evaluation function we proposed requires fewer computational resources, while providing a more effective focusing area and more moderate sensitivity, compared to the traditional evaluation function. In contrast to the traditional climbing method, our proposed fast auto-focusing search algorithm also can spot the focus point in a shorter time, effectively reducing the round trips of the focusing motor and improving the focusing accuracy.

The problem of synthesis of missing image parts represents an interesting and challenging area of image processing and computer vision with significant potential. This thesis, focuses on an adaptive depth-guided image completion method that addresses the image completion problem using information contained in the rest of the image. The completion process is separated into structure and texture synthesis. A method is first introduced for completing the respective depth map through the use of a diffusion-based operation, preserving global image structure within the unknown region. Building upon the state of the art exemplar based inpainting technique of Barnes et al., we complete the target (unknown) region by matching to and blending source patches drawn from the rest of the image, using the reconstructed depth information to guide the completion process. Secondly, for each target patch, we formulate an adaptive patch size determination as an optimization problem that minimizes an objective function involving local image gradient magnitude and orientations. An extension to the coherence- based objective function introduced by Wexler et al. is then introduced, which not only encourages coherence of the respective target region with respect to the source region in colour but also in depth. We further consider the variance between patches in the SSD criteria for preventing error accumulation and propagation. Experimental results show that our method can provide a significant improvement to patch-based image completion algorithms shown by PSNR and SSIM calculations as well as a qualitative subjective study.

Texture synthesis and morphing are important techniques for efficiently creating realistic textures used in scientific and entertainment applications. In this paper we present a novel fast algorithm for multi-dimensional texture synthesis and morphing that is especially suitable for parallel architectures such as GPUs or direct volume rendering (DVR) hardware. Our proposed solution generalizes the synthesis process to support higher than three-dimensional synthesis and morphing.

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce the computation by limiting the number of searching locations. This is accomplished at the expense of less accuracy of motion estimation and gives rise to an appreciably higher mean squared error (MSE) for motion compensated images. We present a new fast and efficient search algorithm for block motion estimation that produces better quality performance and less computational time compared with a three-step search (TSS) algorithm. The proposed algorithms are based on the ideas of the dithering pattern for pixel decimation, multiple-candidate for pixel-decimation-based full search, and center-based distribution of the motion vector. From the experimental results, the proposed algorithm is superior to the TSS in both quality performance (about 0.2 dB) and computational complexity (about half).

Fast search motion estimation algorithms when compared to full search motion estimation algorithms often converge to a local minimum, providing a momentous reduction in computational cost. However, the motion vectors measurement process in fast search algorithms is subject to noise and matching errors. Researchers have investigated the use of mathematical tools used for stochastic estimation from noisy measurements, to seek optimal estimates. Amongst those tools, is the conventional Kalman filtering, which addresses the general problem of trying to estimate the state of a discrete-time controlled process that is governed by the linear stochastic difference. This research investigates the possible combinations of benchmark motion estimation algorithms and the Kalman filter. In this paper, the author presents an in-depth investigation and a detailed analysis on the use of the above combination, and seeks to establish conditions under which the application would be successful. Experimental results show that the above is possible only under certain conditions and constraints of certain properties of the video sequences being coded. Furthermore, a recommendation is made on when it is possible to use the adaptive Kalman filter instead of the conventional filter to enhance the motion vectors at the cost of extra complexity.

The paper provides an intelligent method of texture reconstruction after removal of non-disabled objects or artifacts in video sequences. Data under subtitles, logotypes, damages of information medium or small size objects are referred to as missing data. A novel implementation of separated neural network was used to receive spatial texture estimations in missing data region. Usually several types of textures are located under removed object. A fast wave algorithm was developed for boundary interpolation between different types of texture into a missing data region. Three strategies of wave algorithm for contour optimization were suggested. A fully connected one-level neural network was applied for choice of texture inpainting method (blurring, texture tile, and texture synthesis). The proposed technique was tested for visual reconstruction of missing text regions (subtitles, logotypes) and missing objects with area less 8-12% of frame in animation and movies. In the first case, a simplified decision without stage of boundaries approximation may be applied; in the second case, the reconstruction results are significantly determined by a background complexity and motion in scene.

Transform domain texture synthesis