RGB-D Face Recognition With Texture and Attribute Features

Face recognition research mainly focuses on traditional 2D color images, which is extremely susceptible to be affected by external factors such as various viewpoints and has limited recognition accuracy. In order to achieve improved recognition performance, as well as the 3D face holds more abundant information than 2D, we present a 3D human face recognition algorithm using the Microsoft's Kinect. The proposed approach integrates the depth data with the RGB data to generate 3D face raw data and then extracts feature points, identifies the target via a two-level cascade classifier. Also, we build a 3D-face database including 16 individuals captured exclusively using Kinect. The experimental results indicate that the introduced algorithm can not only achieve better recognition accuracy in comparison to existing 2D and 3D face recognition algorithms when the probe face is exactly in front of Kinect sensor, but also can increase 9.3% of recognition accuracy compared to the PCA-3D algorithm when it is not confronting the camera.

One of the popularly explored topics for the past few decades in the field of artificial intelligence is face recognition. RGB-D picture-based face recognition has attracted numerous researcher’s interest as it has more benefits compared with two-dimensional face recognitions. This review paper gives a brief review of RGB-D face recognition researches. First, this paper reviewed some of the RGB-D sensors available in the market and its specifications. After that, description and scope of face databases which are utilized to test the accuracy of these face recognition techniques are explored. Finally, this paper presents a summary of traditional techniques and deep learning techniques on RGB-D face recognitions.

After decades of extensive research, the field of 2D face recognition is fruitful. However, 2D face recognition is sensitive to changes in pose, facial expression and illumination. 3D face recognition has great potential due to the inherent invariance of pose and illumination changes, but these 3D face images are still difficult to acquire due to several issues such as cost and accessibility. In contrast, RGB-D images captured by low-cost depth sensors such as Kinect are relatively easy to acquire, and research on such low-quality inputs is still limited. In this thesis, an end-to-end multimodal fusion method based on spatial and channel attention is proposed to effectively fuse two image modalities, RGB and depth, to enhance RGB-D face recognition. Our work is specified as follows: firstly, three branches combined with the spatial and channel attention modules are established on the basis of ResNet18, so that the features of three modalities (RGB, depth map, and their fusion modalities, respectively) are obtained. The features under these three modalities are then fused, fed into a shared layer, and the fused features are further learned for deeper discriminative features and passed through a spatial attention vectorization module at the end. The method achieves good results on the Intellifusion RGBD dataset, while we perform ablation trials, which well reveal the contribution of each component of our method to the final performance.

2D Face recognition has been extensively studied for decades and has reached remarkable results in recent years. However, 2D Face recognition is sensitive to variations in poses, facial expressions and illuminations. Depth images provide valuable information to help model facial boundaries and understand the global facial layout and provide low frequency patterns. Intuitively, RGB-D images are more robust to external environments than RGB images. Unfortunately, RGB-D datasets are orders of magnitude smaller than 2D datasets and insufficient to train a deep CNN model as effective as RGB-based models. To tackle these challenges, we present an RGB-D ResNet50 model which can be transferred from a pretrained RGB model and takes RGB-D images as input. We achieved an accuracy of 94.64% and won the \(1^{\text {st}}\) place on 3D Face Recognition Algorithm Challenge, 2019 BenchCouncil International Artificial Intelligence System Challenges.

Face recognition based on 2D images is a difficult problem, because the illumination, pose and expression changes in the images create great statistical differences and the identity of the face itself becomes shadowed by these factors. 3D face recognition has the potential to overcome feature localization, pose and illumination problems, and it can be used in conjunction with 2D systems. We review the relevant work on 3D face recognition here, and discuss merits and demerits of different representations and recognition algorithms. This survey focuses on 3D face data acquisition and 3D face recognition using histogram features. Future works involved in developing more accurate 3D face recognition are also discussed. These include the need for improved 3D face data acquisition, 3D face recognition algorithms and 3D face experimental methodology.

We propose a novel method to improve the performance of existing three dimensional (3D) human face recognition algorithms that employ Euclidean distances between facial fiducial points as features. We further investigate a novel 3D face recognition algorithm that employs geodesic and Euclidean distances between facial fiducial points. We demonstrate that this algorithm is robust to changes in facial expression. Geodesic and Euclidean distances were calculated between pairs of 25 facial fiducial points. For the proposed algorithm, geodesic distances and 'global curvature' characteristics, defined as the ratio of geodesic to Euclidean distance between a pairs of points, were employed as features. The most discriminatory features were selected using stepwise linear discriminant analysis (LDA). These were projected onto 11 LDA directions, and face models were matched using the Euclidean distance metric. With a gallery set containing one image each of 105 subjects and a probe set containing 663 images of the same subjects, the algorithm produced EER=1.4% and a rank 1 RR=98.64%. It performed significantly better than existing algorithms based on principal component analysis and LDA applied to face range images. Its verification performance for expressive faces was also significantly better than an algorithm that employed Euclidean distances between facial fiducial points as features.

Face Recognition is an efficient technique and one of the most liked biometric software application for the identification and verification of specific individual in a digital image by analysing and comparing patterns. This paper presents a survey on well-known techniques of face recognition. The primary goal of this review is to observe the performance of different face recognition algorithms such as SVM (Support Vector Machine), CNN (Convolutional Neural Network), Eigenface based algorithm, Gabor Wavelet, PCA (Principle Component Analysis) and HMM (Hidden Markov Model). It presents comparative analysis about the efficiency of each algorithm. This paper also figure out about various face recognition applications used in real world and face recognition challenges like Illumination Variation, Pose Variation, Occlusion, Expressions Variation, Low Resolution and Ageing in brief. Another interesting component covered in this paper is review of datasets available for face recognition. So, must needed survey of many recently introduced face recognition aspects and algorithms are presented.

Facial recognition is a feature that uses facial detection algorithms to detect a face and then invokes facial recognition algorithms to try to match the person’s face. First, a person’s face needs to be enrolled; during enrollment, the person’s facial details are saved in a database—in this case, a mobile phone. The facial recognition algorithm uses this database to match the currently presented face with the faces saved in the database. The efficiency of a facial recognition algorithm depends upon the speed at which it can detect and recognize faces. The problem we address in this paper is that a reliable, automated method for testing facial recognition features in mobile phones. Because the multimedia capabilities of smartphones have expanded phenomenally, the need for thoroughly testing facial recognition algorithms has become crucial. The uses of facial recognition can range from facial authentication for unlocking phones to security for a number of other applications. To meet these needs, a reliable, automated test for validating the facial recognition algorithms must be developed. The challenge for the software test team was to automate the test cases, which involve tilting the phone at specific angles from the test subject. The permissible angular movements of the phone are determined by the algorithmic specifications of the facial recognition algorithm. We tested scenarios involving multiple faces, motion blur, panning the phone in front of the test subject faces at various speeds, and so on. We adopted a robotic arm to perform the facial recognition test cases and developed software to program the robotic arm to test the phone’s facial recognition software for functionality, performance, and stability as well as adversarial tests. This paper discusses the computer vision use case for facial recognition and describes how we developed an automated facial recognition test suite using a robotic arm.

This is the first study to compare the PCA and ICP approaches to 3D face recognition, and to propose a local region approach coping with expression variation in 3D face recognition. A new algorithm for 3D face recognition is proposed for handling expression variation. It uses a surface registration-based technique for 3D face recognition. The proposed method uses a fully automatic approach to use to initialize the 3D matching. Results are presented for gallery and probe datasets of 355 subjects imaged in 3D, with significant time lapse between gallery and probe images of a given subject yielding 3,205 3D models. We find that an ICP-based method performs better than a PCA-based method. The evaluation results show that our proposed new algorithm substantially improves performance in the case of varying facial expression. We also examined subject factors in the proposed method on 3D face models by age and gender.

We present a new algorithm for 3D face recognition, and compare its performance to that of previous approaches. We focus especially on the case of facial expression change between gallery and probe images. We first establish performance comparisons using a PCA (eigenface) algorithm and an ICP (iterative closest point) algorithm similar to ones reported in the literature. Experimental results show that the performance of either approach degrades substantially in the case Then we introduce a new algorithm, Adaptive Rigid Multi-region Selection, is introduced to independently matches multiple facial regions and creates a fused result. This algorithm is fully automated and used no manually selected landmark points. Experimental results show that our new algorithm substantially improves performance in the case of varying facial expression. Our experimental results are based on the largest 3D face dataset to date, with 449 persons, over 4,000 3D images, and substantial lapse between gallery and probe images.

Attendance marking in a classroom during a lecture is not only a onerous task but also a time consuming one at that. Due to an unusually high number of students present during the lecture there will always be a probability of proxy attendance(s).Attendance marking with conventional methods has been an area of challenge. The growing need of efficient and automatic techniques of marking attendance is a growing challenge in the area of face recognition. In recent years, the problem of automatic attendance marking has been widely addressed through the use of standard biometrics like fingerprint and Radio frequency Identification tags etc., However,these techniques lack the element of reliability. In this proposed project an automated attendance marking and management system is proposed by making use of face detection and recognition algorithms. Instead of using the conventional methods, this proposed system aims to develop an automated system that records the student’s attendance by using facial recognition technology. The main objective of this work is to make the attendance marking and management system efficient, time saving, simple and easy. Here faces will be recognized using face recognition algorithms. The processed image will then be compared against the existing stored record and then attendance is marked in the database accordingly. Compared to existing system traditional attendance marking system, this system reduces the workload of people. This proposed system will be implemented with 4 phases such as Image Capturing, Segmentation of group image and Face Detection, Face comparison and Recognition, Updating of Attendance in database.

In this paper, face recognition algorithms under three low-quality conditions, namely, complex illumination, low resolution and partial occlusion, are summarized, and relatively new face recognition algorithms under low-quality conditions are introduced and explained. Such as the algorithm of face recognition with occlusion based on partition, face recognition algorithm based on Iteratively Reweighted Robust Principal Component and face recognition with occlusion based on MEBML, illumination face recognition based on RPCA and Convolutional Neural Network, face recognition based on Illumination Normalization and Block-based Adaptive Local Ternary Pattern, low-resolution face recognition based on blocking CS-LBP and weighted PCA algorithm, pose Robust low-resolution face recognition via CKEDA and other face recognition algorithm under low-quality conditions. The existing problems and development trend of face recognition under low quality conditions are analyzed.

In this thesis we present two applications where 3D surface modeling and registration play a role of paramount importance. The first application is in archaeology, where our proposed method assists the archaeologist in the tedious process of reconstructing ceramic vessels from the excavated fragments (or shards). The method takes advantage of the surface intrinsic features coupled with a library of generic models constructed by the archaeologists to produce a virtual reconstruction of what the original vessel may have looked like. Generic models are generated based on the expert historical knowledge of the period, provenance of the artifact, and location of the digging site. The generic models need not to be identical to the original vessel, but must be within a geometric transformation of it in most of its parts. The methods are suited for ceramic vessels with distinctive color patterns or reliefs (i.e., surface with molding, carving, or stamping). We introduce a novel curvature B-spline fitting method that makes the extraction of robust features possible. The alignment of the shards against the generic model uses a novel set of 3D weighted curve moments and absolute affine invariants derived from them. The transformation is computed from corresponding parabolic contours on the shard and the generic model. When the shards have no color or relief on them, they are virtually mended to abutting shards using intrinsic differential anchor points computed on the shards break curves and a set of absolute invariants. For axially symmetric vessels, a global constraint induced by the surface of revolution is applied to guarantee global mending consistency. In our experiment involving around 300 shards, more than 85% of them are successfully mended. The mended shards consist of over 90% surface areas of the ceramic vessels. The second application is in 3D human face synthesis and recognition, where our proposed method assists the national security agencies in constructing a 3D face database to be used for spotting missing personnel or identify possible suspects when close up images of that person are flagged up. A gender/race generic 3D face model is morphed into a personalized unique 3D face of a given person from using a limited number of 2D images of the given person taken from different viewpoints. The 2D images need not to be of the same resolution, or taken under similar brightness or same place, or even same time period. The method applies also to drawing sketches of the person in question. The reconstructed 3D personalized model is an accurate rendition of the actual human face, and the reconstruction errors are smaller than a 3D scanner resolution that is used to scan the person's face. Our method renders a difficult 2D face recognition problem into a much simpler 3D surface registration (alignment) problem. Given a set of images of a test person, a personalized 3D face model is constructed and "best" aligned with one of the 3D faces stored in the database. When the minimum alignment errors are below an accepted threshold (lower than the scanner resolution), the identity of the test person is matched with the one in the database. Experimental results are shown in controlled environment with high resolution images, under ideal illumination condition, as well as for images taken in uncontrolled environment with low resolution, arbitrary illumination. In both conditions, the recognition rate is over 95%. It is also proved that better face recognition rate is achieved by using a gender/race based generic 3D face model which allows for accurate pose estimation as well as the face identification in 3D.

Due to its wide applications in practice, face recognition has been an active research topic. With the availability of adequate training samples, many machine learning methods could yield high face recognition accuracy. However, under the circumstance of inadequate training samples, especially the extreme case of having only a single training sample, face recognition becomes challenging. How to deal with conflicting concerns of the small sample size and high dimensionality in one-sample face recognition is critical for its achievable recognition accuracy and feasibility in practice. Being different from the conventional methods for global face recognition based on generalization ability promotion and local face recognition depending on image segmentation, a single-sample face recognition algorithm based on locality preserving projection (LPP) feature transfer is proposed here. First, transfer sources are screened to obtain the selective sample source using the whitened cosine similarity metric. Second, we project the vectors of source faces and target faces into feature subspace by LPP, respectively, and calculate the feature transfer matrix to approximate the mapping relationship on source faces and target faces in subspace. Then, the feature transfer matrix is used on training samples to transfer the original macro characteristics to target macro characteristics. Finally, the nearest neighbor classifier is used for face recognition. Our results based on popular databases FERET, ORL, and Yale demonstrate the superiority of the proposed LPP feature transfer-based one-sample face recognition algorithm when compared with popular single-sample face recognition algorithms, such as (PC) $^{2}\text{A}$ and Block FLDA.

Based on neural network and local binary pattern algorithm, this paper builds a lightweight artificial face recognition system on chip Firefly-RK3399, with high speed, strong robustness and high recognition accuracy. Our embedded artificial intelligent face recognition system mainly consists of face detection, feature extraction and recognition. Multi-task convolutional neural network (MTCNN) under the CaffeOnACL framework is utilized for face detection, and the local binary pattern (LBP) is applied as face recognition algorithm. Experiments illustrate that our artificial intelligent embedded face recognition system has high speed and accuracy, which is easy-carrying and of high commercial value as well.