Facial Image Inpainting with Variational Autoencoder

A bi-modal face recognition framework integrating facial expression with facial appearance

Efficient modeling of high-dimensional data requires extracting only relevant dimensions through feature learning. Unsupervised feature learning has gained tremendous attention due to its unbiased approach, no need for prior knowledge or expensive manual processing, and ability to handle exponential data growth. Deep Autoencoder (AE) is a state-of-the-art deep neural network for unsupervised feature learning, which learns embedded-representations using a series of stacked layers. However, as the AE network gets deeper, these learned embedded-representations can deteriorate due to vanishing gradient, leading to performance degradation. This article presents ResNet Autoencoder (RAE) and its convolutional version (C-RAE) for unsupervised feature learning. The advantage of RAE and C-RAE is that it enables the user to add residual connections for increased network capacity without incurring the cost of degradation for unsupervised feature learning compared to standard AEs. While RAE and C-RAE inherit all the advantages of AEs, such as automated non-linear feature extraction and unsupervised learning, they also allow users to design larger networks without adverse effects on feature learning performance. We performed classification on learned embedded-representation to evaluate RAE and C-RAE. RAE and C-RAE were compared against AEs on MNIST, Fashion MNIST, and CIFAR10 datasets. When increasing the number of layers, C-RAE outperformed AE by showing significantly lower performance degradation of classification accuracy (less than 3%) compared to AE (33% to 65%). Further, C-RAE exhibited higher mean accuracy and lower variance of accuracy than standard AE. When comparing RAE and C-RAE with widely used feature learning methods (Convolutional AE, PCA, ICA, LLE, Factor Analysis, and SVD), C-RAE showed the highest accuracy.

Facial appearance changes because uncontrolled variations of facial appearances due to illumination, pose, expression, occlusion of non-cooperative subjects and subject-to-camera distance need to be handled to allow for successful recognition. This paper presents a novel image quality assessment model. The model is designed to reduce the influence which is caused by the degradation of facial image quality due to uncontrolled variations of facial appearances, and the degradation can lower the recognition performance. The model assesses the image quality from several aspects: (I) Occlusion measure. (II) Face-to-camera distance measure. (Ill) Pose and expression measure.(IV) Uneven illumination measure. Then noisy score is calculated by the image quality assessment model while higher noisy score's images will be discarded for face recognition, the superior face images are selected by image quality assessment model to obtain best recognition result. Experimental results on CAS-PEAL face databases with varied uncontrolled facial appearances demonstrated that the proposed approach achieved satisfactory recognition rate.

Call for Papers: Special Issue ofComputer Vision and Image Understandingon the Analysis of Volumetric Images

Facial age estimation is of interest due to its potential to be applied in many real-life situations. However, recent age estimation efforts do not consider juveniles. Consequently, we introduce a juvenile age detection scheme called LaGMO, which focuses on the juvenile aging cues of facial shape and appearance. LaGMO is a combination of facial landmark points and Term Frequency Inverse Gravity Moment (TF-IGM). Inspired by the formation of words from morphemes, we obtained facial appearance features comprising facial shape and wrinkle texture and represented them as terms that described the age of the face. By leveraging the implicit ordinal relationship between the frequencies of the terms in the face, TF-IGM was used to compute the weights of the terms. From these weights, we built a matrix that corresponds to the possibilities of the face belonging to the age. Next, we reduced the reference matrix according to the juvenile age range (0–17 years) and avoided the exhaustive search through the entire training set. LaGMO detects the age by the projection of an unlabeled face image onto the reference matrix; the value of the projection depicts the higher probability of the image belonging to the age. With Mean Absolute Error (MAE) of 89% on the Face and Gesture Recognition Research Network (FG-NET) dataset, our proposal demonstrated superior performance in juvenile age estimation.

Face recognition has been a major research theme over the last two decades. There are several problems to be solved to improve the performance of face recognition. Such major problems involve appearance variation due to pose, illumination, expression, and aging. In particular, aging includes internal and external factors that cause facial appearance variation and, consequently, it is the most difficult problem to handle. In this paper, we propose a face recognition method that is robust against facial appearance variation due to aging. The proposed method employs segmentation verification of frontal face images that consists of the following three steps. (1) Face image segmentation generates three regional subimages from the input face image. (2) A matching score is calculated using gradient features from a pair consisting of the input image and a registered image for each of the three generated subimages and original (whole face) image. We obtain four matching scores. (3) The verifying classifier evaluates the matching score vector formed of the matching scores calculated for each of the four images and predicts the a posteriori probability that two matching images belong to the same person. The results of an experimental evaluation with the FGNET and MORPH face aging datasets clarify the effectiveness of the proposed method for age invariant face recognition.

The process of aging causes significant alterations in the facial appearance of individuals. When compared with other sources of variation in face images, appearance variation due to aging displays some unique characteristics. Changes in facial appearance due to aging can even affect discriminatory facial features, resulting in deterioration of the ability of humans and machines to identify aged individuals. We describe how the effects of aging on facial appearance can be explained using learned age transformations and present experimental results to show that reasonably accurate estimates of age can be made for unseen images. We also show that we can improve our results by taking into account the fact that different individuals age in different ways and by considering the effect of lifestyle. Our proposed framework can be used for simulating aging effects on new face images in order to predict how an individual might look like in the future or how he/she used to look in the past. The methodology presented has also been used for designing a face recognition system, robust to aging variation. In this context, the perceived age of the subjects in the training and test images is normalized before the training and classification procedure so that aging variation is eliminated. Experimental results demonstrate that, when age normalization is used, the performance of our face recognition system can be improved.

In this paper we propose an auto encoder-based method for the unsupervised identification of subword units. We experiment with different types and architectures of auto encoders to assess what auto encoder properties are most important for this task. We first show that the encoded representation of speech produced by standard auto encoders is more effective than Gaussian posteriorgrams in a spoken query classification task. Finally we evaluate the subword inventories produced by the proposed method both in terms of classification accuracy in a word classification task (with lexicon size up to 263 words) and in terms of consistency between subword transcription of different word examples of a same word type. The evaluation is carried out on Italian and American English datasets.

The discipline of Computer Vision has generated interested in computer scientists, psychologists and neuroscientists. In this, Computer vision has been successfully used in face detection, face recognition, facial expression recognition and emotion identification. The next great step in computer vision is to identify personality traits from facial appearance. From ancient times, it has been proved that it is possible to assess personality traits and types from facial appearance or features of the face. While personality traits can be assessed using psychological or psychometric tests, interviews, and direct observation it is important, as the fields of AI and deep learning advance, that the psychological tests are automated to build and develop digital physiognomy software. The advantages of automatic personality traits assessment from facial appearances are far and wide ranging. This automation can be easily arrived at using deep learning algorithms especially by retraining the Inception v3 CNN model.

Face identity recognition in autism spectrum disorders: A review of behavioral studies

Face recognition (FR) is a less intrusive biometrics technology with various applications, such as security, surveillance, and access control systems. FR remains challenging, especially when there is only a single image per person as a gallery dataset and when dealing with variations like pose, illumination, and occlusion. Deep learning techniques have shown promising results in recent years using VAE and GAN, with approaches such as patch-VAE, VAE-GAN for 3D Indoor Scene Synthesis, and hybrid VAE-GAN models. However, in Single Sample Per Person Face Recognition (SSPP FR), the challenge of learning robust and discriminative features that preserve the subject's identity persists. To address these issues, we propose a novel framework called AD-VAE, specifically for SSPP FR, using a combination of variational autoencoder (VAE) and Generative Adversarial Network (GAN) techniques. The proposed AD-VAE framework is designed to learn how to build representative identity-preserving prototypes from both controlled and wild datasets, effectively handling variations like pose, illumination, and occlusion. The method uses four networks: an encoder and decoder similar to VAE, a generator that receives the encoder output plus noise to generate an identity-preserving prototype, and a discriminator that operates as a multi-task network. AD-VAE outperforms all tested state-of-the-art face recognition techniques, demonstrating its robustness. The proposed framework achieves superior results on four controlled benchmark datasets-AR, E-YaleB, CAS-PEAL, and FERET-with recognition rates of 84.9%, 94.6%, 94.5%, and 96.0%, respectively, and achieves remarkable performance on the uncontrolled LFW dataset, with a recognition rate of 99.6%. The AD-VAE framework shows promising potential for future research and real-world applications.

An efficient face recognition system should recognize faces in different views and poses. The efficiency of a human face recognition system depends on the capability of face recognition in presence of changes in the appearance of face due to expression, pose and illumination. A novel algorithm which utilizes the combination of texture and depth information based on Modular PCA to overcome the problem of pose variation and illumination change for face recognition is proposed. The system has combined 2D and 3D systems in the feature level which presents higher performance in contrast with methods which utilizes either 2D or 3D system separately. A multimodal face recognition based on Modular PCA when compared with conventional PCA algorithm has an improved recognition rate for face images with large variations in illumination and facial expression. The proposed algorithm is tested with FRAV3D database that has faces with pose variation and illumination changes. Recognition rates from experimental results show the superiority of Modular PCA over conventional PCA methods in tackling face images with different pose variations and changes in illuminations. The proposed algorithm shows a recognition rate of 86% that is achieved in fusion experiment.

Classical autoencoders (AE) learn a compressed, meaningful representation of the input data and denoising autoencoders (DAE) capture the true underlying data manifold even when inputs are noisy. Data is the foundation of artificial intelligence, and thus for all autoencoder types. However, all types produce, when well trained, output data which are similar to the input data. This could lead to output data being added to the data that is to be used for further learning. We show on ECG signals that adding AE/DAE-generated reconstructions to the training set — intended to augment data — causes catastrophic performance collapse.

Facial cosmetics have the ability to substantially alter the facial appearance, which can negatively affect the decisions of a face recognition. In addition, it was recently shown that the application of makeup can be abused to launch so-called makeup presentation attacks. In such attacks, the attacker might apply heavy makeup in order to achieve the facial appearance of a target subject for the purpose of impersonation. In this work, we assess the vulnerability of a COTS face recognition system to makeup presentation attacks employing the publicly available Makeup Induced Face Spoofing (MIFS) database. It is shown that makeup presentation attacks might seriously impact the security of the face recognition system. Further, we propose an attack detection scheme which distinguishes makeup presentation attacks from genuine authentication attempts by analysing differences in deep face representations obtained from potential makeup presentation attacks and corresponding target face images. The proposed detection system employs a machine learning-based classifier, which is trained with synthetically generated makeup presentation attacks utilizing a generative adversarial network for facial makeup transfer in conjunction with image warping. Experimental evaluations conducted using the MIFS database reveal a detection equal error rate of 0.7% for the task of separating genuine authentication attempts from makeup presentation attacks.

Face recognition from the real data, capture images, sensor images and database images is challenging problem due to the wide variation of face appearances, illumination effect and the complexity of the image background. Face recognition is one of the most effective and relevant applications of image processing and biometric systems. In this paper we are discussing the face recognition methods, algorithms proposed by many researchers using artificial neural networks (ANN) which have been used in the field of image processing and pattern recognition. How ANN will used for the face recognition system and how it is effective than another methods will also discuss in this paper. There are many ANN proposed methods which give overview face recognition using ANN. Therefore, this research includes a general review of face detection studies and systems which based on different ANN approaches and algorithms. The strengths and limitations of these literature studies and systems were included, and also the performance analysis of different ANN approach and algorithm is analysing in this research study.