Robust face alignment and partial face recognition

Many approaches have been developed for holistic face recognition with impressive performance. However, few studies have addressed the question of how to recognize an arbitrary image patch of a holistic face. In this paper we address this problem of partial face recognition. Partial faces frequently appear in unconstrained image capture environments, particularly when faces are captured by surveillance cameras or handheld devices (e.g. mobile phones). The proposed approach adopts a variable-size description which represents each face with a set of keypoint descriptors. In this way, we argue that a probe face image, holistic or partial, can be sparsely represented by a large dictionary of gallery descriptors. The proposed method is alignment free and we address large-scale face recognition problems by a fast filtering strategy. Experimental results on three public domain face databases (FRGCv2.0, AR, and LFW) show that the proposed method achieves promising results in recognizing both holistic and partial faces.

Component-based face recognition under transfer learning for forensic applications

In this paper, we propose a topology preserving graph matching (TPGM) method for partial face recognition. Most existing face recognition methods extract features from holistic face images, yet faces in real-world unconstrained environments are usually occluded by objects or other faces, which cannot provide the whole face images for recognition. Latest keypoint-based partial face recognition methods only match on the detected keypoints to remove the occluded regions. However, they simply measure the node-wise similarity without higher order geometrical graph information, thereby depending heavily on descriptors which are susceptible to noises. To address this, our TPGM method estimates a non-rigid transformation encoding the second order geometric structure of the graph, so that more accurate and robust correspondence can be computed with the topological information. Experimental results on three widely used face datasets show that the proposed TPGM outperforms most existing state-of-the-art partial face recognition methods.

Face Recognition Technology Based on Partial Facial Features

In recent days, there has been increasing need for recognition of unconstrained face images, such as those collected from the web or captured by mobile devices and video surveillance cameras. In such real world scenarios, human faces could be easily occluded by other objects that make the face recognition task as a complex one. Satisfactory performance has been achieved earlier but often only in the controlled environments. But it is very tedious to obtain holistic face images for unconstrained face recognition. Thus, in order to avoid the degradation of face images and the huge variations due to illumination, pose, occlusion and expression, a new Robust Face Recognition approach is proposed. In this approach, the partial face recognition using Scale Invariant Feature Transform (SIFT) technique is combined with Multi-directional Multi-level Dual Cross Patterns (DCP) technique that makes the recognition task as a robust one when compared to other face recognition approaches. Then, the Robust Point Set Matching (RPSM) is used to match the corresponding stable keypoints from both the gallery image and probe face image. Finally, PNN and K-NN classification are used to classify the face images even with the presence of occlusion, random partial crop, illumination, pose and exaggerated facial expression. The proposed robust face recognition system is evaluated based on the performance parameters such as sensitivity, specificity, accuracy, precision and recall.

In this paper, we present a novel method for performing robust illumination-tolerant and partial face recognition that is based on modeling the phase spectrum of face images. We perform principal component analysis in the frequency domain on the phase spectrum of the face images and we show that this improves the recognition performance in the presence of illumination variations dramatically compared to normal eigenface method and other competing face recognition methods such as the illumination subspace method and fisherfaces. We show that this method is robustly even when presented with partial views of the test faces, without performing any pre-processing and without needing any a-priori knowledge of the type or part of face that is occluded or missing. We show comparative results using the illumination subset of CMU-PIE database consisting of 65 people showing the performance gain of our proposed method using a variety of training scenarios using as little as three training images per person. We also present partial face recognition results that obtained by synthetically blocking parts of the face of the test faces (even though training was performed on the full face images) showing gain in recognition accuracy of our proposed method.

A crucial part of the face recognition process, along with facial attribute analysis and face verification, is facial landmark detection. In order to prepare face images for feature extraction in facial analysis jobs, face alignment is an essential step. Alignment is frequently used in training and inference to standardize the locations of important facial landmarks for applications including face recognition, facial emotion detection, and facial attribute classification. The effectiveness of facial analysis models is known to be greatly impacted by the approach and application of face alignment. However, there has not been enough research done on how alignment affects the quality of facial images. Although face alignment techniques based on convolutional neural networks have advanced significantly recently, occlusion remains a fundamental obstacle to achieving high accuracy. In order to enhance performance, we present in this study the attentioned distillation module from our earlier work on the Occlusion-Adaptive Deep Network model. A distillation module in this model infers the occlusion probability of every place in high-level features. When measuring the relationship between facial shape and look, it can be automatically learned. However, because of the missing semantic features, the holistic face cannot be represented by the clean feature representation. We must use a low-rank learning module to recover lost features in order to achieve exhaustive and complete feature representation.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

These Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Automatic facial expression recognition system has many applications including, but not limited to, human behaviour understanding, detection of mental disorders, and synthetic human expressions. Recognition of facial expression by computer with high recognition rate is still a challenging task. Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classification. The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control

In this paper, we describe a fully automatic video retrieval prototype system that uses an image or a video sequence of an interested identity as probe. The system is based on face vision techniques including face detection and tracking, face alignment and recognition. Given a film or TV sitcom, first face trajectories are extracted in video by head tracking that decompose the video into segments corresponding to certain identity, then frames containing faces of higher quality are selected and normalized according to face alignment results, and finally different segments are associated by face recognition. Experiments are carried out on news video, feature length film video and TV sitcom to show its effectiveness. Potential usage of our system includes intelligent DVD/VCD browsing, video database retrieval, meeting record browsing, etc.

Partial face recognition (PFR) is challenging as the appearance of the face changes significantly with occlusion. In particular, these occlusions can be due to any item and may appear in any position that seriously hinders the extraction of discriminative features. Existing methods deal with PFR either by training a deep model with existing face databases containing limited occlusion types or by extracting un-occluded features directly from face regions without occlusions. Limited training data (i.e., occlusion type and diversity) can not cover the real-occlusion situations, and thus training-based methods can not learn occlusion robust discriminative features. The performance of occlusion region-based method is bounded by occlusion detection. Different from limited training data and occlusion region-based methods, we propose to use multi-label attributes for Partial Face Recognition (Attr4PFR). A novel data augmentation is proposed to solve limited training data and generate occlusion attributes. Apart from occlusion attributes, we also include soft biometric attributes and semantic attributes to explore more rich attributes to combat the loss caused by occlusions. To train our Attr4PFR, we propose an implicit attributes loss combined with a softmax loss to enforce Attr4PFR to learn discriminative features. As multi-label attributes are our auxiliary signal in the training phase, we do not need them in the inference. Extensive experiments on public benchmark AR and IJB-C databases show our method is 3% and 2.3% improvement compared to the state-of-the-art.

Cognitive domains related to attention and executive functions (a set of cognitive processes that regulate, control, and manage other cognitive abilities) seem to influence the recognition of facial expressions in people with Alzheimer's disease (AD). We examined the relationship between facial expression recognition, global cognition and executive function in people with AD according to their cognitive level. In a cross-sectional design, we included 130 participants with AD divided into three groups based on their Mini-Mental State Examination (MMSE) scores: MMSE 1 (scores 23-28), MMSE 2 (scores 17-22), and MMSE 3 (scores 11-16). Facial expression recognition ability was analyzed using the Faces Test. Executive function was analyzed using the Trail Making Test (TMT), the Verbal Fluency Test (VFT), the Semantic Fluency Test (SFT), the Digit Span Forward (DSF) and Backward (DSB) tests, and the Clock Drawing Test (CDT). In MMSE 1 group difficulties in divided attention and cognitive flexibility impacted the accuracy of facial expression recognition. In the MMSE 2 group, facial expression recognition was related to impairment in working memory. In the MMSE 3 group, the impact on facial expression recognition was directly related to visuoconstructive abilities. We observed that the executive resources involved in each evaluated group differed in terms of facial recognition task performance efficacy. Interventions at stimulating executive and visuoconstructive abilities in people with AD may contribute to better preservation of facial expression recognition.

Facial images are of critical importance in many real-world applications from gaming to surveillance. The current literature on facial image analysis, from face detection to face and facial expression recognition, are mainly performed in either RGB, Depth (D), or both of these modalities. But, such analyzes have rarely included Thermal (T) modality. This paper paves the way for performing such facial analyzes using synchronized RGB-D-T facial images by introducing a database of 51 persons including facial images of different rotations, illuminations, and expressions. Furthermore, a face recognition algorithm has been developed to use these images. The experimental results show that face recognition using such three modalities provides better results compared to face recognition in any of such modalities in most of the cases.

Studies on two-factor authentication based on RFID and face recognition have been carried out on a large scale. However, these studies didn't discuss the way to overcome the weaknesses of face recognition authentication in the access control systems. In this study, two authentication factors, RFID and face recognition, were implemented using the LBP (Local Binary Pattern) algorithm to overcome weaknesses of face recognition authentication in the access control system. Based on the results of performance testing, the access control system has 100% RFID authentication and 80% face recognition authentication. The average time for the RFID authentication process is 0.03 seconds, the face recognition process is 6.3885 seconds and the verification of the face recognition is 0.1970 seconds. The access control system can still work properly after three days without being switched off. The results of security testing showed that the capabilities spoofing detection has 100% overcome the photo attack.

Face recognition has many important applications in areas such as public surveillance and security, identity verification in the digital world, and modeling techniques in multimedia data management. Facial expression recognition is also important for targeted marketing, medical analysis, and human–robot interaction. In this paper, we survey a few techniques for facial analysis. We compare the cloud platform AWS Rekognition, convolutional neural networks, transfer learning from pre-trained neural nets, and traditional feature extraction using facial landmarks for this analysis. Although not comprehensive, this survey covers a lot of ground in the state-of-the-art solutions for facial analysis. We show that to get high accuracy, good-quality data and processing power must be provided in large quantities. We present the results of our experiments which have been conducted over six different public as well as proprietary image data sets.