2D-3D Pose Invariant Face Recognition System for Multimedia Applications

In this paper, a noble nonintrusive three-dimensional (3D) face modeling system for random-profile-based 3D face recognition is presented. Although recent two-dimensional (2D) face recognition systems can achieve a reliable recognition rate under certain conditions, their performance is limited by internal and external changes, such as illumination and pose variation. To address these issues, 3D face recognition, which uses 3D face data, has recently received much attention. However, the performance of 3D face recognition highly depends on the precision of acquired 3D face data, while also requiring more computational power and storage capacity than 2D face recognition systems. In this paper, we present a developed nonintrusive 3D face modeling system composed of a stereo vision system and an invisible near-infrared line laser, which can be directly applied to profile-based 3D face recognition. We further propose a novel random-profile-based 3D face recognition method that is memory-efficient and pose-invariant. The experimental results demonstrate that the reconstructed 3D face data consists of more than 50 k 3D point clouds and a reliable recognition rate against pose variation.

This paper presents a novel face recognition approach which uses only partial information in the recognition stage. The algorithm is based on an extension of the classical PCA and is called partial PCA (P2CA). The P/sup 2/CA is a combined 2D-3D scheme which requires 3D face data in the training process but can process 2D pictures in the recognition stage. The strategy has been proven to be very robust in pose variation scenarios showing that the 3D training process retains all the spatial information of the face while the 2D picture effectively recovers the face information from the available data. Simulation results with a multi-view face database have shown recognition rates above 92% when using 180/spl deg/ texture face images in the training stage and 2D face pictures taken from different angles (from -90/spl deg/ to +90/spl deg/) in the recognition stage.

The accuracy of 2D face recognition (FR) has progressed significantly due to the availability of large-scale training data. However, the research of deep learning based 3D FR is still in the early stage. Most of available 3D FR generate 2D maps from 3D data and apply existing 2D CNNs to the generated 2D maps for feature extraction. We propose in this paper a light-weight framework, named PointFace, to directly process point set data for 3D FR. In this framework, two weight-shared encoders are designed to directly extract features from a pair of 3D faces and the distances between embeddings of the same person and different person are minimized and maximized, respectively. The framework also use a feature similarity loss to guide the encoders to obtain discriminative face representations. A pair selection strategy is proposed to generate positive and negative face pairs to further improve the FR performance. Extensive experiments on Lock3DFace and Bosphorus show that the proposed PointFace outperforms state-of-the-art 2D CNN based FR methods.

Since 3D data has additional depth information than 2D image, it can be used various researches and applications such as robot vision, car navigation and human computer interaction. Especially, 3D face recognition is robust against facial pose and light variation since it can use 3D face shape information such as facial profile, curvature and depth map. However, it is difficult to acquire 3D face data in real-time since conventional 3D data acquisition systems like laser scanner or structured light system are very slow for the data capturing. Recently, Time-of-Flight (ToF) camera provides full range distance data at high framerates and thus comes into the spotlight as an alternative to conventional 3D acquisition systems. ToF camera provides both distance images and gray-scale images simultaneously in realtime. However, it has drawbacks which are low resolution and gray-scale image, not color image. In this paper, a registration method of color and ToF cameras is presented to find precise corresponding color pixel information with range distance data from ToF camera.

Abstract3D shape similarity seach have been studied for detecting or finding a specified 3D model among 3D CAD model database. We propose to use 3D point data for the search, because it has become easier to obtain 3D point data by photographs. CAD data is converted into 3D point data in advance. Then, using machine learning, we attempt to match those data with the 3D point data acquired in the field. It can be expected that the accuracy of matching is improved by directly handling 3D data. As a preliminary trial, we have tried to clasify 10 kinds of chair models in represented as 3D point data with a machine learning approach. It was suggested that 3D shape matching between 3D point data is possible by our proposed method as the result.

Face recognition based on 2D images is a difficult problem, because the illumination, pose and expression changes in the images create great statistical differences and the identity of the face itself becomes shadowed by these factors. 3D face recognition has the potential to overcome feature localization, pose and illumination problems, and it can be used in conjunction with 2D systems. We review the relevant work on 3D face recognition here, and discuss merits and demerits of different representations and recognition algorithms. This survey focuses on 3D face data acquisition and 3D face recognition using histogram features. Future works involved in developing more accurate 3D face recognition are also discussed. These include the need for improved 3D face data acquisition, 3D face recognition algorithms and 3D face experimental methodology.

This research sought to examine differences in the cross-modal facilitation effect of voice on facial identity recognition under mask occlusion for both oneself and others. Employing a facial recognition paradigm, we examined the influence of voice on facial identity recognition under static and dynamic mask occlusion through two eye-tracking experiments. The behavioral results from Experiments 1 and 2 indicate that mask occlusion interfered with recognition for both static and dynamic faces, with greater interference observed for others' faces than for self-faces. In addition, voice exerts cross-modal enhancement effects on faces, with greater enhancement observed for masked faces than for no mask. Furthermore, voice provides stronger enhancement for others' dynamic faces than for their self-dynamic faces. Eye-tracking data from both experiments revealed that the difference in dynamic facial recognition between self-faces and others' faces due to voice emerged in the early stages of dynamic facial recognition and persisted into later stages. However, regardless of whether they were in the early or late stages of static facial recognition, the facilitation effect of voice did not differ between themselves and others. This study revealed that the cross-modal facilitation of visual stimuli by voice is influenced by the self-advantage effect.

The fast development of modern technologies has led to the frequent usage of face detection and recognition approaches in various fields due to their user-friendly nature. Generally, face recognition approaches identify individuals by their facial features. Recently, numerous face recognition methods were designed by leveraging the significance of deep learning approaches. These approaches are divided into numerous tasks, and implementing these tasks is time consuming as they require real-time images for processing. However, facial features vary across time because of ageing, haircuts, etc. Furthermore, images with poor resolution may also influence the recognition ability of the framework. Therefore, an advanced face recognition framework is designed in this work. The key intent of this designed model is to tackle the obstacles of the classical models and detect individuals in diverse environmental circumstances with better accuracy. Here, the relevant images are collected from a standard data source, and the quality of the image is enhanced using a Retinex-based image enhancement technique. Next, the Generative Adversarial Network (GAN) is employed to synthesize real images that mitigate the effects of pose discrepancies and occlusion, providing clearer features for recognition. Next, synthesized images are fed to Residual network 50 (ResNet50) for face recognition. Finally, the precise face segmentation is done with the support of Gather-Excite Networks of ResUNet (GEN-ResUNet). This integrated approach enhances the accuracy of face recognition, especially in challenging environments with lighting variations and occlusions. The effectiveness of the model is proven by conducting extensive experimentation.

Face recognition research mainly focuses on traditional 2D color images, which is extremely susceptible to be affected by external factors such as various viewpoints and has limited recognition accuracy. In order to achieve improved recognition performance, as well as the 3D face holds more abundant information than 2D, we present a 3D human face recognition algorithm using the Microsoft's Kinect. The proposed approach integrates the depth data with the RGB data to generate 3D face raw data and then extracts feature points, identifies the target via a two-level cascade classifier. Also, we build a 3D-face database including 16 individuals captured exclusively using Kinect. The experimental results indicate that the introduced algorithm can not only achieve better recognition accuracy in comparison to existing 2D and 3D face recognition algorithms when the probe face is exactly in front of Kinect sensor, but also can increase 9.3% of recognition accuracy compared to the PCA-3D algorithm when it is not confronting the camera.

In this paper, we developed a family of 2D and 3D invariant features with applications to 3D human faces recognition. The main contributions of this paper are: (a) systematically deriving a family of novel features, called summation invariant that are invariant to Euclidean transformation in both 2D and 3D; (b) developing an effective method to apply summation invariant to the 3D face recognition problem. Tested with the 3D data from the face recognition grand challenge v1.0 dataset, the proposed new features exhibit achieves a performance that rivals the best 3D face recognition algorithms reported so far

Our work is motivated by the problem of automatic face recognition, a difficult task, still missing a general solution. Its complexity lies in the wide range of variations presents in the input data, due to different lightings, background scenes and head positions. Moreover, the face appearance is affected by internal sources of variations: on a long temporal scale aging and weight gain, and on a short scale the action of the facial muscles. An effective recognition algorithm should be insensitive to all these sources of variations. During the last decade, good results to the recognition problem have been obtained using 3D Morphable Models (3DMMs). Their use allowed to separate the data variations due to the identity from the ones due to external sources like the lighting conditions. However, other internal sources were not considered. Our goal is to include expressions as an additional source of internal variation in 3DMMs, enabling us to recognize faces not only under different illuminations and pose conditions, but also with different expressions. In general, the construction of a 3DMM requires a corpus of training data; for our task we need a training set including examples of both identity and expression variations. Unfortunately, their acquisition alone is not sufficient, since they have to be previously registered with a reference 3D head model. The registration of 3D scans of expressions is a difficult problem, which could not be solved with the registration algorithm previously used. The main contribution of our work is a new registration algorithm which can cope with arbitrary expressions in the 3D data. Our algorithm is also capable of registering data with missing values, an important property since virtually no 3D acquisition devices is immune to holes and artifacts in the output. Given the training set of registered 3D examples, we construct a 3DMM where identity and expression variations are represented with two separate linear Gaussian models. The two models are then linearly combined, yielding an expression-identity 3DMM which we apply to the problem of 3D face recognition. Although this modeling approach does not take into account the interdependency between expressions and identity, the recognition performance is not negatively affected.

We present a promising analysis on using the pattern of symmetry in the face to increase the accuracy of three-dimensional face recognition. We introduce the concept of the dasiaaverage-half-facepsila, motivated by the symmetry preserving singular value decomposition. We compare face recognition results using the eigenfaces face recognition algorithm with average-half-face data and full face data in several experiments on a 3D face data set of 1126 images. We show that the results from the eigenfaces face recognition system using the average-half-face is more accurate than using the full face, only the left or right half of the face or a random choice of half of the face.

Face recognition is a challenging task as it has to deal with several issues such as illumination orientation and variability among the different faces. Previous works have shown that 3D face is a robust biometric trait and is less sensitive to light and pose variations. Also due to availability of inexpensive sensors and new 3D data acquisition techniques it has become easy to capture 3D data. A 3D depth image of a face is found to be rich in information and biometric recognition performance can be enhanced by using 3D face data along with convolutional neural network. However the shortcoming of this approach is the conversion of 3D data to lower dimensions (depth image) which suffer from loss of geometric information and the network becomes computationally expensive. In this work we endeavor to apply deep learning method for 3D face recognition and propose a deep convolutional neural network based on PointNet architecture which consumes point cloud directly as input and siamese network for similarity learning. Further we propose a solution to the issue of a limited database by applying data augmentation at the point cloud level. Our proposed technique shows encouraging performance on Bosphorus and IIT Indore 3D face databases.

A comprehensive account of almost all the feature extraction methods used in current face recognition systems is presented. Specifically we have made distinction in the holistic and local feature extraction and differentiate them qualitatively as opposed to quantitatively. It is argued that a global feature representation should be preferred over a bag-of-feature approach. The problems in current feature extraction techniques and their reliance on a strict alignment is discussed. Finally we have introduced to use face-GLOH signatures that are invariant with respect to scale, translation and rotation and therefore do not require properly aligned images. The resulting dimensionality of the vector is also low as compared to other commonly used local features such as Gabor, LBP etc. and therefore learning based methods can also benefit from it. In a typical multi-view face recognition task, where it is assumed to have several examples of a subject available for training, we have shown in an extensive experimental setting the advantages and weaknesses of commonly used feature descriptions. Our results show that under more realistic assumptions, most of the classifiers failed on conventional features. While using the introduced face-GLOH-signature representation is relatively less affected by large in-class variations. This has been demonstrated by providing a fair performance comparison of several classifiers under more practical conditions such as misalignments, large number of subjects and large pose variations. An important conclusion is to be drawn from the results on FERET is that conventional multi-view face recognition cannot cope well with regards to large pose variations. Even using a large number of training examples in different poses for a subject do not suffice for a satisfactory recognition. In order to solve the problem where only one training example per subject is available, many recent methods propose to use image synthesis to generate a given subject at all other views and then perform a conventional multi-view recognition (Beymer and Poggio, 1995; Gross et al, 2004). Besides the fact that such synthesis techniques cause severe artefacts and thus cannot preserve the identity of an individual, a conventional classification cannot yield good recognition results, as has been shown in an extensive experimental setting. More sophisticated methods are therefore needed in order to address pose invariant face recognition. Large pose differences cause significant appearance variations that in general are larger than the appearance variation due to identity. One possible way of addressing this is to learn these variations across each pose, more specifically by fixing the pose and establishing a correspondence on how a person’s appearance changes under this pose one could reduce the in-class appearance variation significantly. In our very recent work (Sarfraz and Hellwich, 2009), we demonstrate the usefulness of face-GLOH signature in this direction.

Research on 3D face recognition has been intensified in recent years. However, most research has focused on the 3D static data analysis. In this paper, we investigate the face recognition problem using dynamic 3D face model sequences. Based on our newly created 3D dynamic face database, we propose to use a spatio-temporal hidden Markov model (HMM) which incorporates 3D surface feature characterization to learn the spatial and temporal information of faces. The advantage of using 3D dynamic data for face recognition has been evaluated by comparing our approach to three conventional approaches: 2D video based temporal HMM model, conventional 2D-texture based approach (e.g., Gabor wavelet based approach), and static 3D-model-based approaches.