Dictionary Integration Using 3D Morphable Face Models for Pose-Invariant Collaborative-Representation-Based Classification

In this paper, a novel framework for 3D face reconstruction from a single 2D face image was proposed. We focus on generating 3D face model without expensive devices and complicated calculation. First, we preprocess 2D face image, including illumination compensation, face detection and feature point extraction. Then, the method is based on a 3D morphable face model that encodes shape and texture in terms of model parameters. The prior 3D face model is a linear combination of “eigenheads” obtained by applying PCA to a training set of laser-scanned 3D faces. To account for pose and illumination variations, the algorithm simulates the process of image formation in 3D face and it estimates 3D face and texture of faces from a single image. As opposed to previous literature, our method can locate facial feature points automatically in a 2D facial image. Moreover, we also show that our proposed method for pose estimation can be successfully applied to 3D face reconstruction. In our experiment results, the method proposed has a satisfied performance regarding calculating time and the reconstructing photorealistic 3D faces.

Collaborative representation based face classification exploiting block weighted LBP and analysis dictionary learning

3D face modeling from 2D face images is of significant importance for face analysis, animation and recognition. Previous research on this topic mainly focused on 3D face modeling from a single 2D face image; however, a single face image can only provide a limited description of a 3D face. In many applications, for example, law enforcement, multi-view face images are usually captured for a subject during enrollment, which makes it desirable to build a 3D face texture model, given a pair of frontal and profile face images. We first determine the correspondence between un-calibrated frontal and profile face images through facial landmark alignment. An initial 3D face shape is then reconstructed from the frontal face image, followed by shape refinement utilizing the depth information provided by the profile image. Finally, face texture is extracted by mapping the frontal face image on the recovered 3D face shape. The proposed method is utilized for 2D face recognition in two scenarios: (i) normalization of probe image, and (ii) enhancing the representation capability of gallery set. Experimental results comparing the proposed method with a state-of-the-art commercial face matcher and densely sampled LBP on a subset of the FERET database show the effectiveness of the proposed 3D face texture model.

This paper proposes a 3D face fitting algorithm based on the 3D Morphable Face Model. This algorithm updates parameters at each stage in the fitting process. To simplify measurement of parameters, this algorithm updates the parameters in cylindrical coordinates. The parameters are updated using a cost function which is the difference between the input 3D face data and the fitted 3D face model. This proposed algorithm shows good results when shape, texture and extrinsic variations occur in the 3D domain. The average correlation of the fitted shape and texture parameters to the test values are 0.72 and 0.99, respectively. This 3D face fitting algorithm can be widely used for 3D face analysis and 3D face recognition.

Example‐based statistical face models using principle component analysis (PCA) have been widely deployed for three‐dimensional (3D) face reconstruction and face recognition. The two common factors that are generally concerned with such models are the size of the training dataset and the selection of different examples in the training set. The representational power (RP) of an example‐based model is its capability to depict a new 3D face for a given 2D face image. The RP of the model can be increased by correspondingly increasing the number of training samples. In this contribution, a novel approach is proposed to increase the RP of the 3D face reconstruction model by deforming a set of examples in the training dataset. A PCA‐based 3D face model is adapted for each new near frontal input face image to reconstruct the 3D face shape. Further an extended Tikhonov regularisation method has been employed to reconstruct 3D face shapes from a set of facial points. The results justify that the proposed adaptive PCA‐based model considerably improves the RP of the standard PCA‐based model and outperforms it with a 95% confidence level.

In this paper, a noble nonintrusive three-dimensional (3D) face modeling system for random-profile-based 3D face recognition is presented. Although recent two-dimensional (2D) face recognition systems can achieve a reliable recognition rate under certain conditions, their performance is limited by internal and external changes, such as illumination and pose variation. To address these issues, 3D face recognition, which uses 3D face data, has recently received much attention. However, the performance of 3D face recognition highly depends on the precision of acquired 3D face data, while also requiring more computational power and storage capacity than 2D face recognition systems. In this paper, we present a developed nonintrusive 3D face modeling system composed of a stereo vision system and an invisible near-infrared line laser, which can be directly applied to profile-based 3D face recognition. We further propose a novel random-profile-based 3D face recognition method that is memory-efficient and pose-invariant. The experimental results demonstrate that the reconstructed 3D face data consists of more than 50 k 3D point clouds and a reliable recognition rate against pose variation.

Recent researches in three-dimensional (3D) face recognition area have proved that, 3D face recognition methods, achieve better accuracy than its 2D counterpart. One of the main advantages of 3D face recognition methods is that it measures the geometry of rigid features on the human face, due to which it becomes an invariant to illumination, expressions and rotations of head (pose variation). But the main drawback of 3D face recognition methods is the acquisition of 3D face images which generally needs a range camera. It is where the research community is facing difficulty. Research in 3D face recognition also depends on available databases. In this paper, we present a qualitative review of all the available 3D face databases. A comparative table of all the databases is also given at the end with all the necessary parameters.

3D face reconstruction from a single image is an important task in many multimedia applications. Recent works typically learn a CNN-based 3D face model that regresses coefficients of a 3D Morphable Model (3DMM) from 2D images to perform 3D face reconstruction. However, the shortage of training data with 3D annotations considerably limits performance of these methods. To alleviate this issue, we propose a novel 2D-Assisted Learning (2DAL) method that can effectively use “in the wild” 2D face images with noisy landmark information to substantially improve 3D face model learning. Specifically, taking the sparse 2D facial landmark heatmaps as additional information, 2DAL introduces four novel self-supervision schemes that view the 2D landmark and 3D landmark prediction as a self-mapping process, including the landmark self-prediction consistency for 2D and 3D faces respectively, cycle-consistency over the 2D landmark prediction and self-critic over the predicted 3DMM coefficients based on landmark prediction. Using these four self-supervision schemes, 2DAL significantly relieves the demands for the the conventional paired 2D-to-3D annotations and gives much higher-quality 3D face models without requiring any additional 3D annotations. Experiments on AFLW2000-3D, AFLW-LFPA and Florence benchmarks show that our method outperforms state-of-the-arts for both 3D face reconstruction and dense face alignment by a large margin.

2D face images represent faces with incomplete information. 3D face reconstruction from a single 2D image is a challenging problem with application value. The single feature extraction method distorts the generated 3D face image. In this paper, we use contour-based face segmentation method to reconstruct 3D face image. We focus on the edge and contour information of the face when using contour lines to segment the face. Different from the global 3D face reconstruction method, we combine the global and local face information to carry out 3D face reconstruction. Our method: First of all, we do contour segmentation for human faces and extract the features of the segmented images. Second, we learn the local binary features of each keypoint in a complete face image, then combine the features and use linear regression to detect the keypoints. Thirdly, we use Convolutional Neural Networks to learn the regression 3D Morphable Model coefficient and significantly improve the quality and efficiency of reconstruction. We regressed the coefficients of the 3D deformable model from 2D images to present face alignment for 3D face reconstruction. We carry out feature mapping between 2D face and 3D face image, and monitor and verify 3D face model through mapping relationship. Our method can not only reconstruct face images from all angles, but also reduce face deformities. We made face images fit better under different expressions and postures.

This report describes the large-scale experimental results from the Face Recognition Vendor Test (FRVT) 2006 and the Iris Challenge Evaluation (ICE) 2006. The FRVT 2006 looks at recognition from high-resolution still images and three-dimensional (3D) face images, and measures performance for still images taken under controlled and uncontrolled illumination. The ICE 2006 reports iris recognition performance from left and right iris images. The FRVT 2006 results from controlled still images and 3D images document an order-of-magnitude improvement in recognition performance over the FRVT 2002. This order-of-magnitude improvement was one of the goals of the preceding technology development effort, the Face Recognition Grand Challenge (FRGC). The FRVT 2006 and the ICE 2006 compared recognition performance from very-high resolution still face images, 3D face images, and single-iris images. On the FRVT 2006 and the ICE 2006 datasets, recognition performance was comparable for all three biometrics. In an experiment comparing human and algorithm performance, the best-performing face recognition algorithms were more accurate than humans. These and other results are discussed in detail.

The use of three-dimensional (3D) facial scans for facial analysis is increasing in maxillofacial treatment. The aim of this study was to investigate the consistency of two-dimensional (2D) and 3D facial analyses performed by multiple raters. Six men and four women (25–36-year-old) participated in this study. The 2D images of the smiling and resting faces in the frontal and sagittal planes were obtained. The 3D facial and intraoral scans were merged to generate virtual 3D faces. Ten clinicians performed facial analyses by investigating 14 indices of 2D and 3D faces. Intra- and interrater agreements of the results of 2D and 3D facial analyses within and among the participants were evaluated. The intrarater agreement between the 2D and 3D facial analyses varied according to the indices. The highest and lowest agreements were found for the dental crowding index (0.94) and smile line curvature index (0.56) in the frontal plane, and Angle's classification (canine) index (0.98) and occlusal plane angle index (0.55) in the profile plane. In the frontal plane, the interrater agreements were generally higher for the 3D images than for the 2D images, while in the profile plane, the interrater agreements were high in the Angle's classification (canine) index however low in the other indices. Several occlusion-related indices were missing in the 2D images because the posterior teeth were not observed. Esthetic analysis results between 2D and 3D face images can differ according to the evaluation indices. The use of 3D faces is recommended over 2D images to increase the reliability of facial analyses, as it can fully assess both esthetic and occlusion-related indices.

Unlike 2D face images, obtaining a 3D face is not easy. Existing methods, therefore, create a 3D face from a 2D face image (3D face reconstruction). A user might wish to edit the reconstructed 3D face, but 3D face editing has seldom been studied. This paper presents such method and shows that reconstruction and editing can help each other. In the presented framework named NEO-3DF, the 3D face model we propose has independent sub-models corresponding to semantic face parts. It allows us to achieve both local intuitive editing and better 3D-to-2D alignment. Each face part in our model has a set of controllers designed to allow users to edit the corresponding features (e.g., nose height). In addition, we propose a differentiable module for blending the face parts and making it possible to automatically adjust the face parts (both the shapes and the locations) so that they are better aligned with the original 2D image. Experiments show that the results of NEO-3DF outperform existing methods in intuitive face editing and have better 3D-to-2D alignment accuracy (14% higher IoU) than global face model-based reconstruction. Code available at https://github.com/ubc-3d-vision-lab/NEO-3DF .

T3D face reconstruction from a single 2D image is mathematically ill-posed. However, to solve ill-posed problems in the area of computer vision, a variety of methods has been proposed; some of the solutions are to estimate latent information or to apply model based approaches. In this paper, we propose a novel method to reconstruct a 3D face from a single 2D face image based on pose estimation and a deformable model of 3D face shape. For 3D face reconstruction from a single 2D face image, it is the first task to estimate the depth lost by 2D projection of 3D faces. Applying the EM algorithm to facial landmarks in a 2D image, we propose a pose estimation algorithm to infer the pose parameters of rotation, scaling, and translation. After estimating the pose, much denser points are interpolated between the landmark points by a 3D deformable model and barycentric coordinates. As opposed to previous literature, our method can locate facial feature points automatically in a 2D facial image. Moreover, we also show that the proposed method for pose estimation can be successfully applied to 3D face reconstruction. Experiments demonstrate that our approach can produce reliable results for reconstructing photorealistic 3D faces.

Reconstruction of a 3D face from a single 2D face image can be substantially useful in many areas such as security, forensic, 3D animation, and motion capture. Most 3D face reconstruction techniques require multiple 2D face images taken at different views in order to estimate the depth of each face component which is one of the most crucial parameters in 3D face reconstruction. We propose a new method that is capable of estimating the depth order of a 2D face image in various illumination conditions. The proposed method uses Hillcrest-Valley classification with adaptive mean filter and Otsu threshold value selection to estimate the depth order of an image. The proposed method was evaluated on the YaleB face database consists of 10 individual frontal faces for the total of 576 face images and the experimental results reveal that the proposed method is robust and it can estimate the depth order of a frontal face up to as many as 50 layers regardless of lighting condition.

During the last decade, sparse representations have been successfully applied to design high-performing classification algorithms such as the classical sparse representation based classification (SRC) algorithm. More recently, collaborative representation based classification (CRC) has emerged as a very powerful approach, especially for face recognition. CRC takes advantage of SRC through the notion of collaborative representation, relying on the observation that the collaborative property is more crucial for classification than the l1-norm sparsity constraint on coding coefficients used in SRC. This paper follows the same general philosophy of CRC and its main novelty is the application of a virtual collaborative projection (VCP) routine designed to train images of every class against the other classes to improve fidelity before the projection of the query image. We combine this routine with a method of local feature extraction based on high-order statistical moments to further improve the representation. We demonstrate using extensive experiments of face recognition and classification that our approach performs very competitively with respect to state-of-the-art classification methods. For instance, using the AR face dataset, our method reaches 100% of accuracy for dimensionality 300.