Iterative Refinement of Alpha Matte using Binary Classifier

Pre-processing is an important step in digital image matting, which aims to classify more accurate foreground and background pixels from the unknown region of the input three-region mask (Trimap). This step has no relation with the well-known matting equation and only compares color differences between the current unknown pixel and those known pixels. These newly classified pure pixels are then fed to the matting process as samples to improve the quality of the final matte. However, in the research field of image matting, the importance of pre-processing step is still blurry. Moreover, there are no corresponding review articles for this step, and the quantitative comparison of Trimap and alpha mattes after this step still remains unsolved. In this paper, the necessity and the importance of pre-processing step in image matting are firstly discussed in details. Next, current pre-processing methods are introduced by using the following two categories: static thresholding methods and dynamic thresholding methods. Analyses and experimental results show that static thresholding methods, especially the most popular iterative method, can make accurate pixel classifications in those general Trimaps with relatively fewer unknown pixels. However, in a much larger Trimap, there methods are limited by the conservative color and spatial thresholds. In contrast, dynamic thresholding methods can make much aggressive classifications on much difficult cases, but still strongly suffer from noises and false classifications. In addition, the sharp boundary detector is further discussed as a prior of pure pixels. Finally, summaries and a more effective approach are presented for pre-processing compared with the existing methods.

With the development of digital multimedia technologies, image matting has gained increasing interests from both academic and industrial communities. The purpose of image matting is to precisely extract the foreground objects with arbitrary shapes from an image or a video frame for further editing. It is generally known that image matting is inherently an ill-posed problem because we need to output three images out of only one input image. In this paper, we provide a comprehensive survey of the existing image matting algorithms and evaluate their performance. In addition to the blue screen matting, we systematically divide all existing natural image matting methods into four categories: 1) color sampling-based; 2) propagation-based; 3) combination of sampling-based and propagation-based; and 4) learning-based approaches. Sampling-based methods assume that the foreground and background colors of an unknown pixel can be explicitly estimated by examining nearby pixels. Propagation-based methods are instead based on the assumption that foreground and background colors are locally smooth. Learning-based methods treat the matting process as a supervised or semisupervised learning problem. Via the learning process, users can construct a linear or nonlinear model between the alpha mattes and the image colors using a training set to estimate the alpha matte of an unknown pixel without any assumption about the characteristics of the testing image. With three benchmark data sets, the various matting algorithms are evaluated and compared using several metrics to demonstrate the strengths and weaknesses of each method both quantitatively and qualitatively. Finally, we conclude this paper by outlining the research trends and suggesting a number of promising directions for future development.

Alpha matting is known as a process of accurately separating a foreground object from an image and it has been extensively studied in both academic and industrial areas as an important part of multimedia technology. However, most existing methods utilize single image to get its alpha matte. In this paper, we propose a novel chroma keying method based on stereo images, which can be applied to post-process the alpha matte generated by any matting approach. Given a pair of stereo images, a new matting Laplacian matrix is constructed based on the affinities between matching pixels and their neighbors from two frames. Based on the new matting Laplacian matrix, a new cost function is also formulated to estimate alpha values of the reference image through the propagation between stereo images. The experimental results illustrate that this method can be applied to different categories of existing alpha matting approaches and can obtain alpha matte of higher quality.

Image matting is a process of extracting a foreground object from a complex background. This paper proposes a robust interactive image matting approach. The method requires only a few user interactions in the form of drawing a rectangle and a few strokes to indicate background and foreground. We consider the constraints of accuracy and continuity for the estimated alpha values together to find the optimal matte by iteratively energy optimization. Different from existing sampling-based natural image matting methods which use only intensity information from statistic sampling of known foreground and background pixels to estimate the unknown pixels. We consider the distribution of the known pixels in color, texture and spatial spaces, and build a more robust statistical model. At each iteration, the statistical model is updated according to previous results of matting. Furthermore an accuracy function of sampling is proposed. These manipulations make the sampling of foreground and background pixels more accurate and thus improve the performance of the matting processing. Experiments show that compared with previous approaches, our method is more efficient to extract high quality matte for texture-rich images and difficult images in which foreground and background have very similar colors, while requiring a surprising small amount of user interaction.

Trimap plays an essential role in image matting. However, generating trimaps is expensive. Instead, we propose an architecture named A-U-Net equipped with both spatial and channel attention modules to estimate alpha matte without trimaps. However, there are two opposite phenomena observed in estimating alpha matte. The fine-structure edges connecting to the background should be preserved, whereas the texture details inside the foreground should be ignored. Thus, alpha matte is prone to making errors when the foreground is cluttered with texture or when both the foreground and the background have similar textures. To alleviate this problem, we design an additional classifier to learn the pure foreground from the input image. This “pure” indicates that those pixels must be foreground ones. Then, we add it to the original A-U-Net architecture result and successfully solve the problem. Extensive experiments have proved that our method is comparable or even better than other state-of-the-art methods.

Alpha matting aims to estimate the translucency of an object in a given image. The resulting alpha matte describes pixel-wise to what amount foreground and background colors contribute to the color of the composite image. While most methods in literature focus on estimating the alpha matte, the process of estimating the foreground colors given the input image and its alpha matte is often neglected, although foreground estimation is an essential part of many image editing workflows. In this work, we propose a novel method for foreground estimation given the alpha matte. We demonstrate that our fast multi-level approach yields results that are comparable with the state-of-the-art while outperforming those methods in computational runtime and memory usage.

Contrasts in neuropsychological test profile between first episode patients with schizophrenia or affective disorders

We propose a new method to extract foreground edges in a video streams taken from a stationary camera. Our background model is based on the fact that a background pixel's gradient components follow Gaussian mixture model(GMM). GMM is performed on the initial group of video frames to obtain the initial pixel gradient component distribution information at each pixel. Then each of the current Canny edge pixels is classified into foreground or background pixel based on its gradient components' weighted square sum of distances from their respective mean values. If the difference is larger than a threshold, it is then classified as a foreground pixel, otherwise a background pixel in which case the GMM information is accordingly updated. If the ratio of the number of foreground pixels over the total number of Canny edge pixel is large than a certain threshold, a new GMM background modeling is trigger. The algorithm is implemented in Visual C++ and tested on a laptop powered by an Intel Pentium 3.0GHz. The experiment shows the algorithm is highly selective in extracting valid foreground edge pixels and it's speed is 43 ms/frame for a video stream of 640×480 and shows that the method is applicable for real-time processing.

A case study on computer-aided diagnosis of nonerosive reflux disease using deep learning techniques

Determining optimal bead central angle by applying machine learning to wire arc additive manufacturing (WAAM)

Disparity estimation belongs to the most important, but difficult, problems in image processing and computer vision. Its importance stems from a wide range of applications, while its difficulty is related to ill-posedness. To date, numerous disparity estimation algorithms have been developed. In this paper, we consider a particular case of disparity estimation based on two views and a known alpha channel partitioning each view into foreground and background. The main idea is to use this partitioning in order to enhance disparity estimation in the foreground object close to its boundary. We propose a block-based disparity model with two alpha-channel constraints: a photometric one, disabling invalid intensity/color matches, and a geometric one, preventing disparity smoothing between foreground and background. We incorporate these constraints into a Bayesian framework using the maximum a posteriori probability criterion. We experimentally demonstrate improvements in the estimated disparities at foreground object boundaries, and show examples of image relighting using these disparities

In topographic maps, It is difficult to separate the linear elements, including contour lines, roads, latitude and longitude lines from complicated background due to the pixels with aliasing and false colors, and there exists some background in the result images extracted by the existing methods, especially when the color and energy of linear elements and background are similar in some particular maps, or the maps have low contrast contour lines. To solve these problems, this paper introduces the idea of seed spreading, and puts forward a novel method for separating linear elements. In this method, all the seeds carry the color information of the pixels and the energy information in the negative grayscale images, and they can search other pixels as their brothers to be combined into seed groups according to the color and energy similarity. The seeds have good perception of the environment around them, and the shapes of the seed groups are variable. Furthermore, the seeds are determined as linear elements by analyzing the color and energy differences between the seed groups and the areas around them. The experimental results show that our method can distinguish linear elements from the background more accurately than the previous methods.

This paper presents an interactive algorithm for segmentation of natural images. The task is formulated as a problem of spline regression, in which the spline is derived in Sobolev space and has a form of a combination of linear and Green's functions. Besides its nonlinear representation capability, one advantage of this spline in usage is that, once it has been constructed, no parameters need to be tuned to data. We define this spline on the user specified foreground and background pixels, and solve its parameters (the combination coefficients of functions) from a group of linear equations. To speed up spline construction, K-means clustering algorithm is employed to cluster the user specified pixels. By taking the cluster centers as representatives, this spline can be easily constructed. The foreground object is finally cut out from its background via spline interpolation. The computational complexity of the proposed algorithm is linear in the number of the pixels to be segmented. Experiments on diverse natural images, with comparison to existing algorithms, illustrate the validity of our method.

Video matting, or layer extraction, is a classic inverse problem in computer vision that involves the extraction of foreground objects, and the alpha mattes that describe their opacity, from a set of images. Modem approaches that work with natural backgrounds often require user-labelled "trimaps" that segment each image into foreground, background and unknown regions. For long sequences, the production of accurate trimaps can be time consuming. In contrast, another class of approach depends on automatic background extraction to automate the process, but existing techniques do not make use of spatiotemporal consistency, and cannot take account of operator hints such as trimaps. This paper presents a method inspired by natural image statistics that cleanly unifies these approaches. A prior is learnt that models the relationship between the spatiotemporal gradients in the image sequence and those in the alpha mattes. This is used in combination with a learnt foreground colour model and a prior on the alpha distribution to help regularize the solution and greatly improve the automatic performance of such systems. The system is applied to several real image sequences that demonstrate the advantage that the unified approach can afford.

Image matting and segmentation, which are used to extract a foreground object from the background, are primary techniques for digital image and video editing. In digital matting, the transparency of the foreground object is considered, while segmentation performs a rigid extraction of the object. Recently, several algorithms for matting and segmentation problems have been proposed and have provided high-quality results. In this paper, we propose a unified formulation for image matting in the framework of the method of quantification IV based on a review of the previous studies from this framework. Our method also utilizes discriminative information provided by a user (a few strokes drawn by the user). The experimental results show a favorable matting performance.