Face Alignment Based on 3D Face Shape Model and Markov Random Field

Recent researches in three-dimensional (3D) face recognition area have proved that, 3D face recognition methods, achieve better accuracy than its 2D counterpart. One of the main advantages of 3D face recognition methods is that it measures the geometry of rigid features on the human face, due to which it becomes an invariant to illumination, expressions and rotations of head (pose variation). But the main drawback of 3D face recognition methods is the acquisition of 3D face images which generally needs a range camera. It is where the research community is facing difficulty. Research in 3D face recognition also depends on available databases. In this paper, we present a qualitative review of all the available 3D face databases. A comparative table of all the databases is also given at the end with all the necessary parameters.

Robust face alignment and tracking by combining local search and global fitting

This paper presents an algorithm for automatically detecting and segmenting a moving object from a monocular video. Detecting and segmenting a moving object from a video with limited object motion is challenging. Since existing automatic algorithms rely on motion to detect the moving object, they cannot work well when the object motion is sparse and insufficient. In this paper, we present an unsupervised algorithm to learn object color and locality cues from the sparse motion information. We first detect key frames with reliable motion cues and then estimate moving sub-objects based on these motion cues using a Markov Random Field (MRF) framework. From these sub-objects, we learn an appearance model as a color Gaussian Mixture Model. To avoid the false classification of background pixels with similar color to the moving objects, the locations of these sub-objects are propagated to neighboring frames as locality cues. Finally, robust moving object segmentation is achieved by combining these learned color and locality cues with motion cues in a MRF framework. Experiments on videos with a variety of object and camera motion demonstrate the effectiveness of this algorithm.

Cascaded regression has recently become the method of choice for solving non-linear least squares problems such as deformable image alignment. Given a sizeable training set, cascaded regression method learns a set of generic rules that are sequentially applied to solve the least squares problem. Despite the success of cascaded regression for problems such as face alignment and head pose estimation, there are several shortcomings arising in the strategies proposed thus far. For example, the result is sensitive to the initial shape and the storage of the model is too large, and no robustness to self-occlusion. This paper proposes an approach to solve that problem. First, we reconstruct the 3D face from a 2D face image with arbitrary poses and expressions. Second, we project the 3D face shape model to the 2D plane, then we can get a 2D face shape. Third, we use the projected 2D face shape as the initial shape for the cascaded regression instead of the generic mean shape. Comparing to the existing methods the proposed method can increase the accuracy of the face alignment under confirming appropriate performance.

Markov random field (MRF) based methods have been widely used in high spatial resolution (HSR) image classification. However, many existing MRF-based methods put more emphasis on pixel level contexts while less on superpixel level contextual information. To cope with this issue, this article presents a novel bilevel contextual MRF framework, named BLC-MRF, for HSR imagery classification. Specifically, pixel and superpixel level dependence are incorporated into the proposed MRF model to fully exploit spectral–spatial contextual information and preserve object boundaries in HSR images. In BLC-MRF, a pixel level MRF model is first performed and then cascaded as an input of a superpixel level MRF. In superpixel level, unary and pairwise potential terms are constructed by using the superpixel probability estimation method and spectral histogram distance, respectively. At last, a contextual MRF model is conducted and the final classification map can be computed by using $\alpha$ -expansion algorithm. The benefits of BLC-MRF are twofold: first, the pixel and superpixel level contextual information can be exploited under MRF framework to preserve object boundaries for improving the classification performance, and, second, the algorithm can provide promising results with a small number of training samples. Experimental results on three HSR datasets demonstrate that the proposed approach outperforms several state-of-the-art methods in terms of the classification performance.

Proceedings of the 2011 joint ACM workshop on Human gesture and behavior understanding

We propose a hierarchical approach to spatial modeling based on Markov Random Fields (MRF) and multi-resolution algorithms in image analysis. Unlike their geostatistical counterparts that simultaneously specify distributions across the entire field, MRFs are based on a collection of full conditional distributions that rely on local neighborhood of each element. This critical focus on local specification provides several advantages: (a) MRFs are computationally tractable and are ideally suited to simulation-based computation such as MCMC (Markov Chain Monte Carlo) methods, and (b) model extensions to account for non-stationarity, discontinuity and varying spatial properties at various scales of resolution are easily accessible in the MRF framework. Our proposed method is computationally efficient and well suited to reconstruct fine scale spatial fields from coarser, multi-scale samples (e.g., based on seismic and production data) and sparse fine scale conditioning data (e.g., well data). It is easy to implement and can account for the complex, non-linear interactions between different scales as well as precision of the data at various scales in a consistent fashion. We illustrate our method using a variety of examples that demonstrate the power and versatility of the proposed approach. Finally, a comparison with Sequential Gaussian Simulation with Block Kriging (SGSBK) indicates similar performance with less restrictive assumptions.

T3D face reconstruction from a single 2D image is mathematically ill-posed. However, to solve ill-posed problems in the area of computer vision, a variety of methods has been proposed; some of the solutions are to estimate latent information or to apply model based approaches. In this paper, we propose a novel method to reconstruct a 3D face from a single 2D face image based on pose estimation and a deformable model of 3D face shape. For 3D face reconstruction from a single 2D face image, it is the first task to estimate the depth lost by 2D projection of 3D faces. Applying the EM algorithm to facial landmarks in a 2D image, we propose a pose estimation algorithm to infer the pose parameters of rotation, scaling, and translation. After estimating the pose, much denser points are interpolated between the landmark points by a 3D deformable model and barycentric coordinates. As opposed to previous literature, our method can locate facial feature points automatically in a 2D facial image. Moreover, we also show that the proposed method for pose estimation can be successfully applied to 3D face reconstruction. Experiments demonstrate that our approach can produce reliable results for reconstructing photorealistic 3D faces.

It is not only interesting to predict how an individual of a relatively young age will look in the future but also to reconstruct the facial appearance in the past during childhood. It can be even more desirable when different circumstances, behavior and lifestyle and their impacts on the facial shape appearance as a consequence are taken into account. Such may be applicable for many practical reasons in healthcare, forensics psychology, missing people and children, etc. This paper presents the 3D Face Time Machine Matrix (FT2M), a 3D Dynamic Shape Model which is a fusion of two models of ageing and rejuvenation with facial shape variations due to lifestyle and behavioral factors. This dynamic model is learned from a database of three dimensional facial images which is built by ten individual age groups between 3 to 75 years old. 3D facial aging modeling is a complex process since it affects both the shape and texture of the face. We propose a Dynamic face model to transform the given input face to his youthful or adulthood appearance by taking into account his lifestyle and behavioral traits and the probable changes may occur in perceptible appearance by altering its shape and texture simultaneously.

Objective: Physiognomy has over 3000 years of history in China, where the belief that personality can be discerned through physiognomy is widespread. However, it hasn’t been fully verified by scientific research. Through experiments, this paper explores the relationship between face shape and corresponding personality in physiognomy, and how face shape affects people’s judgment of personality. Method: According to the eight face shapes theory of physiognomy, 10 trained laboratory assistants have selected 64 typical faces through 3816 pieces of ID photos following a designated procedure, and tested the selected 64 persons’ scores of Cattell’s 16 Personality Factors Test. Eight more ID photos have been randomly selected, and each one has been modified by Image Processing Technology into eight face shapes, keeping other facial features same to ensure that the only variable is face shape, and ultimately obtained 64 artificial faces. 949 undergraduates, as participants, have visually judged these 128 faces in a laboratory by using E-prime 2.0 and 16PF Rating Scale. Results: Overall, there was no significant difference of tested sixteen personality traits among eight typical faces. Through a post-hoc test, some face shapes are perceived to have certain significant differences in some personality traits than a certain face shape. For example, on factor Q2 of 16PF, a heart-shaped face (M = 2.625*) is significantly lower than a diamond-shaped face (M = 4.375). In contrast, there are various differences among the eight face shapes on people’s visual judgmental of personality traits. For example, the heart-shaped face (M = 4.01**) is significantly lower than all other face shapes on factor A). By comprising the tested personality traits and perceived personality traits of each face shape, there are significant differences among some personality traits (e.g. diamond face on factor B, t = ?2.847**). Conclusions: Traditional physiognomy theory which explains personality by face shapes can’t be supported by the results. People are affected by the inherent stereotype (such as people with square face look like more right-minded), and tend to make a judgment about people’s personalities according to stereotypes of face shapes. Although their judgments are inconformity with the real personality traits, it indeed influences many people’s judgments on personality. According to this research, if people can tailor their face shape to someone’s preferences by using makeup, it will be easier for them to make a good impression with that person.

This paper describes in detail about 2D and 3D face and video face databases developed in the year from 1992 to 2009. The still face database discussed here includes: Our Laboratory Database of Faces (ORL), Yale face database, Yale face databaseB(B stands for Big), Yale face database extendedB, FERET(Face Recognition Technology) face database and CASPEAL is a large scale Chinese face database with varying PEAL(Pose Expression Accessory, and Lighting). The 3D face databases like MIT-CBCL (Centre for Biological and Computational Learning) face recognition database, Georgia Tech face database, Cohn-Kanade Action Unit Face database and 3D BFM (Basel Face Model) are also discussed in this paper. In CK AU, CK+AU(Cohn-Kanade Action Unit) coded facial expression database, a 3D model is developed from a 2D model which is obtained from the captured video of different subjects. The other Face databases and development of new Face databases are discussed at the end of this paper. The application of proposed concept is to develop a new small size face database which includes all the important criteria included in the previous face and video face databases.

In this paper, we present an improved active shape model (ASM) for facial feature extraction. The original ASM method developed by Cootes et al. highly relies on the initialization and the representation of the local structure of the facial features in the image. We use color information to improve the ASM approach for facial feature extraction. The color information is used to localize the centers of the mouth and the eyes to assist the initialization step. Moreover, we model the local structure of the feature points in the RGB color space. Besides, we use 2D affine transformation to align facial features that are perturbed by head pose variations. In fact, the 2D affine transformation compensates for the effects of both head pose variations and the projection of 3D data to 2D. Experiments on a face database of 50 subjects show that our approach outperforms the standard ASM and is successful in facial feature extraction

Scale selection for anisotropic diffusion filter by Markov random field model

Head pose and facial feature detection are important for face analysis. However, many studies reported good results in constrained environment, the performance could be decreased due to the high variations in facial appearance, poses, illumination, occlusion, expression and make-up. In this paper, we propose a hierarchical regression approach, Dirichlet-tree enhanced random forests (D-RF) for face analysis in unconstrained environment. D-RF introduces Dirichlet-tree probabilistic model into regression RF framework in the hierarchical way to achieve the efficiency and robustness. To eliminate noise influence of unconstrained environment, facial patches extracted from face area are classified as positive or negative facial patches, only positive facial patches are used for face analysis. The proposed hierarchical D-RF works in two iterative procedures. First, coarse head pose is estimated to constrain the facial features detection, then the head pose is updated based on the estimated facial features. Second, the facial feature localization is refined based on the updated head pose. In order to further improve the efficiency and robustness, multiple probabilitic models are learned in leaves of the D-RF, i.e. the patch’s classification, the head pose probabilities, the locations of facial points and face deformation models (FDM). Moreover, our algorithm takes a composite weight voting method, where each patch extracted from the image can directly cast a vote for the head pose or each of the facial features. Extensive experiments have been done with different publicly available databases. The experimental results demonstrate that the proposed approach is robust and efficient for head pose and facial feature detection.

The study of face alignment has been an area of intense research in computer vision, with its achievements widely used in computer graphics applications. The performance of various face alignment methods is often image-dependent or somewhat random because of their own strategy. This study aims to develop a method that can select an input image with good face alignment results from many results produced by a single method or multiple ones. The task is challenging because different face alignment results need to be evaluated without any ground truth. This study addresses this problem by designing a feasible feature extraction scheme to measure the quality of face alignment results. The feature is then used in various machine learning algorithms to rank different face alignment results. Our experiments show that our method is promising for ranking face alignment results and is able to pick good face alignment results, which can enhance the overall performance of a face alignment method with a random strategy. We demonstrate the usefulness of our ranking-enhanced face alignment algorithm in two practical applications: face cartoon stylization and digital face makeup.