Random-Profiles-Based 3D Face Recognition System

Recent researches in three-dimensional (3D) face recognition area have proved that, 3D face recognition methods, achieve better accuracy than its 2D counterpart. One of the main advantages of 3D face recognition methods is that it measures the geometry of rigid features on the human face, due to which it becomes an invariant to illumination, expressions and rotations of head (pose variation). But the main drawback of 3D face recognition methods is the acquisition of 3D face images which generally needs a range camera. It is where the research community is facing difficulty. Research in 3D face recognition also depends on available databases. In this paper, we present a qualitative review of all the available 3D face databases. A comparative table of all the databases is also given at the end with all the necessary parameters.

Face recognition is a challenging task as it has to deal with several issues such as illumination orientation and variability among the different faces. Previous works have shown that 3D face is a robust biometric trait and is less sensitive to light and pose variations. Also due to availability of inexpensive sensors and new 3D data acquisition techniques it has become easy to capture 3D data. A 3D depth image of a face is found to be rich in information and biometric recognition performance can be enhanced by using 3D face data along with convolutional neural network. However the shortcoming of this approach is the conversion of 3D data to lower dimensions (depth image) which suffer from loss of geometric information and the network becomes computationally expensive. In this work we endeavor to apply deep learning method for 3D face recognition and propose a deep convolutional neural network based on PointNet architecture which consumes point cloud directly as input and siamese network for similarity learning. Further we propose a solution to the issue of a limited database by applying data augmentation at the point cloud level. Our proposed technique shows encouraging performance on Bosphorus and IIT Indore 3D face databases.

In this thesis we present two applications where 3D surface modeling and registration play a role of paramount importance. The first application is in archaeology, where our proposed method assists the archaeologist in the tedious process of reconstructing ceramic vessels from the excavated fragments (or shards). The method takes advantage of the surface intrinsic features coupled with a library of generic models constructed by the archaeologists to produce a virtual reconstruction of what the original vessel may have looked like. Generic models are generated based on the expert historical knowledge of the period, provenance of the artifact, and location of the digging site. The generic models need not to be identical to the original vessel, but must be within a geometric transformation of it in most of its parts. The methods are suited for ceramic vessels with distinctive color patterns or reliefs (i.e., surface with molding, carving, or stamping). We introduce a novel curvature B-spline fitting method that makes the extraction of robust features possible. The alignment of the shards against the generic model uses a novel set of 3D weighted curve moments and absolute affine invariants derived from them. The transformation is computed from corresponding parabolic contours on the shard and the generic model. When the shards have no color or relief on them, they are virtually mended to abutting shards using intrinsic differential anchor points computed on the shards break curves and a set of absolute invariants. For axially symmetric vessels, a global constraint induced by the surface of revolution is applied to guarantee global mending consistency. In our experiment involving around 300 shards, more than 85% of them are successfully mended. The mended shards consist of over 90% surface areas of the ceramic vessels. The second application is in 3D human face synthesis and recognition, where our proposed method assists the national security agencies in constructing a 3D face database to be used for spotting missing personnel or identify possible suspects when close up images of that person are flagged up. A gender/race generic 3D face model is morphed into a personalized unique 3D face of a given person from using a limited number of 2D images of the given person taken from different viewpoints. The 2D images need not to be of the same resolution, or taken under similar brightness or same place, or even same time period. The method applies also to drawing sketches of the person in question. The reconstructed 3D personalized model is an accurate rendition of the actual human face, and the reconstruction errors are smaller than a 3D scanner resolution that is used to scan the person's face. Our method renders a difficult 2D face recognition problem into a much simpler 3D surface registration (alignment) problem. Given a set of images of a test person, a personalized 3D face model is constructed and "best" aligned with one of the 3D faces stored in the database. When the minimum alignment errors are below an accepted threshold (lower than the scanner resolution), the identity of the test person is matched with the one in the database. Experimental results are shown in controlled environment with high resolution images, under ideal illumination condition, as well as for images taken in uncontrolled environment with low resolution, arbitrary illumination. In both conditions, the recognition rate is over 95%. It is also proved that better face recognition rate is achieved by using a gender/race based generic 3D face model which allows for accurate pose estimation as well as the face identification in 3D.

This paper proposes a 3D face recognition approach using sphere depth image, which is robust to pose variations in unconstrained environments. The input 3D face point clouds is first transformed into sphere depth images, and then represented as a 3DLBP image to enhance the distinctiveness of smooth and similar facial depth images. An improved SIFT algorithm is applied in the following matching process. The improved SIFT algorithm employs the learning to rank approach to select the keypoints with higher stability and repeatability instead of manually rule-based method used by the original SIFT algorithm. The proposed face recognition method is evaluated on CASIA 3D face database. And the experimental results show our approach has superior performance than many existing methods for 3D face recognition and handles pose variations quite well.

In this work, we propose a 3D occlusion facial recognition network based on a multi-feature combination threshold (MFCT-3DOFRNet). First, we design and extract the depth information of the 3D face point cloud, the elevation, and the azimuth angle of the normal vector as new 3D facially distinctive features, so as to improve the differentiation between 3D faces. Next, we propose a multi-feature combinatorial threshold that will be embedded at the input of the backbone network to implement the removal of occlusion features in each channel image. To enhance the feature extraction capability of the neural network for missing faces, we also introduce a missing face data generation method that enhances the training samples of the network. Finally, we use a Focal-ArcFace loss function to increase the inter-class decision boundaries and improve network performance during the training process. The experimental results show that the method has excellent recognition performance for unoccluded faces and also effectively improves the performance of 3D occlusion face recognition. The average Top-1 recognition rate of the proposed MFCT-3DOFRNet for the Bosphorus database is 99.52%, including 98.94% for occluded faces and 100% for unoccluded faces. For the UMB-DB dataset, the average Top-1 recognition rate is 95.08%, including 93.41% for occluded faces and 100% for unoccluded faces. These 3D face recognition experiments show that the proposed method essentially meets the requirements of high accuracy and good robustness.

Face recognition based on 2D images is a difficult problem, because the illumination, pose and expression changes in the images create great statistical differences and the identity of the face itself becomes shadowed by these factors. 3D face recognition has the potential to overcome feature localization, pose and illumination problems, and it can be used in conjunction with 2D systems. We review the relevant work on 3D face recognition here, and discuss merits and demerits of different representations and recognition algorithms. This survey focuses on 3D face data acquisition and 3D face recognition using histogram features. Future works involved in developing more accurate 3D face recognition are also discussed. These include the need for improved 3D face data acquisition, 3D face recognition algorithms and 3D face experimental methodology.

Face recognition research mainly focuses on traditional 2D color images, which is extremely susceptible to be affected by external factors such as various viewpoints and has limited recognition accuracy. In order to achieve improved recognition performance, as well as the 3D face holds more abundant information than 2D, we present a 3D human face recognition algorithm using the Microsoft's Kinect. The proposed approach integrates the depth data with the RGB data to generate 3D face raw data and then extracts feature points, identifies the target via a two-level cascade classifier. Also, we build a 3D-face database including 16 individuals captured exclusively using Kinect. The experimental results indicate that the introduced algorithm can not only achieve better recognition accuracy in comparison to existing 2D and 3D face recognition algorithms when the probe face is exactly in front of Kinect sensor, but also can increase 9.3% of recognition accuracy compared to the PCA-3D algorithm when it is not confronting the camera.

3D face recognition has lately been attracting ever increasing attention. In this paper we review the full spectrum of 3D face processing technology, from sensing to recognition. The review covers 3D face modelling, 3D to 3D and 3D to 2D registration, 3D based recognition and 3D assisted 2D based recognition. The fusion of 2D and 3D modalities is also addressed. The paper complements other reviews in the face biometrics area by focusing on the sensor technology, and by detailing the efforts in 3D face modelling and 3D assisted 2D face matching.

In this paper, a novel framework for 3D face reconstruction from a single 2D face image was proposed. We focus on generating 3D face model without expensive devices and complicated calculation. First, we preprocess 2D face image, including illumination compensation, face detection and feature point extraction. Then, the method is based on a 3D morphable face model that encodes shape and texture in terms of model parameters. The prior 3D face model is a linear combination of “eigenheads” obtained by applying PCA to a training set of laser-scanned 3D faces. To account for pose and illumination variations, the algorithm simulates the process of image formation in 3D face and it estimates 3D face and texture of faces from a single image. As opposed to previous literature, our method can locate facial feature points automatically in a 2D facial image. Moreover, we also show that our proposed method for pose estimation can be successfully applied to 3D face reconstruction. In our experiment results, the method proposed has a satisfied performance regarding calculating time and the reconstructing photorealistic 3D faces.

This paper presents an efficient 3D face recognition method to handle facial expression. The proposed method uses the Surfaces Empirical Mode Decomposition (SEMD), facial curves and local shape descriptor in a matching process to overcome the distortions caused by expressions in faces. The basic idea is that, the face is presented at different scales by SEMD. Then the isometric-invariant features on each scale are extracted. After that, the geometric information is obtained on the 3D surface in terms of radial and level facial curves. Finally, the feature vectors on each scale are associated with their corresponding geometric information. The presented method is validated on GavabDB database resulting a rank 1 recognition rate (RR) of 98.9% for all faces with neutral and non-neutral expressions. This result outperforms other 3D expression-invariant face recognition methods on the same database.

Since 3D data has additional depth information than 2D image, it can be used various researches and applications such as robot vision, car navigation and human computer interaction. Especially, 3D face recognition is robust against facial pose and light variation since it can use 3D face shape information such as facial profile, curvature and depth map. However, it is difficult to acquire 3D face data in real-time since conventional 3D data acquisition systems like laser scanner or structured light system are very slow for the data capturing. Recently, Time-of-Flight (ToF) camera provides full range distance data at high framerates and thus comes into the spotlight as an alternative to conventional 3D acquisition systems. ToF camera provides both distance images and gray-scale images simultaneously in realtime. However, it has drawbacks which are low resolution and gray-scale image, not color image. In this paper, a registration method of color and ToF cameras is presented to find precise corresponding color pixel information with range distance data from ToF camera.

Compared with 2D face recognition systems, 3D facial recognition has been a popular area of study in academia and industry, because of its ability to detect human faces in an unconstrained environment more efficiently. This study presents a new 3D face identification method that uses 3D facial images to compute geodesic distances using 3D landmarks. Feature extraction is computationally more efficient than Euclidean distance using geodesic distance. This study aims to demonstrate how to differentiate identical twins based solely on facial features by developing an efficient algorithm for calculating geodesic paths and distances. We employed the A*, Dijkstra, and fast marching (FM) algorithms to calculate the geodesic distance. Quantitative similarity metrics are obtained and then utilized as inputs in numerous classification methods: random forest (RF), extra tree classifier (ETC), light gradient boosting machine (LGBM), support vector machine (SVM), and bagging classifiers. An expression challenge dataset for 3D twins (3D-TEC), the most challenging data set for 3D face recognition research at Notre Dame University, was used to validate and test the performance of these algorithms. The recall, accuracy, precision, training time, and F1 score were compared for each classifier to determine the best performance. The results from the experiment indicate that, when compared to other algorithms and classification models, the Extra Tree Classifier is the most accurate (by 90%) in predicting face images, whereas A* is the best technique for computing geodesic distance.

Is online store become more and more popular and important in our daily life, online payment method is becoming more and more important too. People need a stable, rapid and secure payment method. This paper expounds how a rapid 3D human face recognition method is used for online payment, which only needs take one photo of face rather than does a slow scan to get 3D face data. It is stable, rapid and secure.

In the past few years, 3D face modeling has gained significant attention. Reliable modeling of the face is necessary for good performance of a 3D assisted face recognition system. In this paper, we model 3D human face from binocular stereo images. The stereo matching problem has been formulated as an energy minimization problem that progressively propagates depth from reliable regions to unreliable regions by an annealing scheme. The concept of smooth 2D grid is used to enable regularizing the final depth solution. In the energy equation, area based matching is used for the data term. The smoothness term relies on annealing. The 2D grid facilitates the estimation of the smoothness parameter, from a correlation profile obtained from neighborhood of the grid node. It is evident from 3D modeling results that the proposed algorithm performs well on images of human faces.

Research on 3D face recognition has been intensified in recent years. However, most research has focused on the 3D static data analysis. In this paper, we investigate the face recognition problem using dynamic 3D face model sequences. Based on our newly created 3D dynamic face database, we propose to use a spatio-temporal hidden Markov model (HMM) which incorporates 3D surface feature characterization to learn the spatial and temporal information of faces. The advantage of using 3D dynamic data for face recognition has been evaluated by comparing our approach to three conventional approaches: 2D video based temporal HMM model, conventional 2D-texture based approach (e.g., Gabor wavelet based approach), and static 3D-model-based approaches.