Multi-subregion based probabilistic approach toward pose-invariant face recognition

An efficient face recognition system should recognize faces in different views and poses. The efficiency of a human face recognition system depends on the capability of face recognition in presence of changes in the appearance of face due to expression, pose and illumination. A novel algorithm which utilizes the combination of texture and depth information based on Modular PCA to overcome the problem of pose variation and illumination change for face recognition is proposed. The system has combined 2D and 3D systems in the feature level which presents higher performance in contrast with methods which utilizes either 2D or 3D system separately. A multimodal face recognition based on Modular PCA when compared with conventional PCA algorithm has an improved recognition rate for face images with large variations in illumination and facial expression. The proposed algorithm is tested with FRAV3D database that has faces with pose variation and illumination changes. Recognition rates from experimental results show the superiority of Modular PCA over conventional PCA methods in tackling face images with different pose variations and changes in illuminations. The proposed algorithm shows a recognition rate of 86% that is achieved in fusion experiment.

This paper represents a new approach for face recognition that incorporates Prewitt edge detection, Gabor filter and Zernike moments to transform the image into a unified domain. On this joint domain, five distance metrics are constructed using Schoenberg transform for the purpose of defining efficient similarity measures for holistic face recognition. The proposed Schoenberg similarity applies Schoenberg transform to the logarithm of five existing distance metrics: Minkowski, City-Block, Euclidean, Soergel and Lorentzian metrics. The constructed Schoenberg logarithmic metrics are called SL-Minkowski, SL-City-Block, SL-Euclidean, SL-Soergel and SL-Lorentzian. These distance metrics are utilized as similarity measures after being normalized over the range [0,1] for fair comparison with existing measures. The proposed Schoenberg system can resist three problems: Change in illumination, pose and facial expression. Simulation results show that the proposed distance measures have superior performance as compared to the classical metrics: Structural Similarity Index Measure (SSIM) and Feature-based Similarity Measure (FSIM). Performance criteria are the recognition rate and the recognition confidence, defined as the similarity difference between the best match and the second-best match in the face database.

In many face recognition tasks the pose and illumination conditions of the probe and gallery images are different. In other cases multiple gallery or probe images may be available, each captured from a different pose and under a different illumination. We propose a face recognition algorithm which can use any number of gallery images per subject captured at arbitrary poses and under arbitrary illumination, and any number of probe images, again captured at arbitrary poses and under arbitrary illumination. The algorithm operates by estimating the Fisher light-field of the subject’s head from the input gallery or probe images. Matching between the probe and gallery is then performed using the Fisher light-fields.

After decades of extensive research, the field of 2D face recognition is fruitful. However, 2D face recognition is sensitive to changes in pose, facial expression and illumination. 3D face recognition has great potential due to the inherent invariance of pose and illumination changes, but these 3D face images are still difficult to acquire due to several issues such as cost and accessibility. In contrast, RGB-D images captured by low-cost depth sensors such as Kinect are relatively easy to acquire, and research on such low-quality inputs is still limited. In this thesis, an end-to-end multimodal fusion method based on spatial and channel attention is proposed to effectively fuse two image modalities, RGB and depth, to enhance RGB-D face recognition. Our work is specified as follows: firstly, three branches combined with the spatial and channel attention modules are established on the basis of ResNet18, so that the features of three modalities (RGB, depth map, and their fusion modalities, respectively) are obtained. The features under these three modalities are then fused, fed into a shared layer, and the fused features are further learned for deeper discriminative features and passed through a spatial attention vectorization module at the end. The method achieves good results on the Intellifusion RGBD dataset, while we perform ablation trials, which well reveal the contribution of each component of our method to the final performance.

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods.

The panorama of face recognition is to find the dissimilarity features that can be used to discriminate individuals for their recognition. The holistic face recognition methods available in the literature perform well in the controlled environments. These methods may not appropriate to identify people through their faces in uncontrolled environments such as changes in pose, facial expression and illumination. However, the feature based methods address the issues of face recognition in uncontrolled environments to a certain extent. Therefore, the biometric researchers are trying to devise the face recognition methods that perform optimally in uncontrolled environments. This paper presents a novel method of face recognition in uncontrolled environments that works with local binary pattern of facial images and compute the dissimilarity among them using Bray Curtis dissimilarity metric. A novel concept of filtering the LBP surface texture is developed. The proposed method we called as "augmented local binary" pattern ((ALBP)) works on a combination of the principle of locality of uniform and non-uniform patterns. It replaces non-uniform patterns with the majority value of uniform patterns and combined with neighboring uniform patterns to extract valuable information regarding local descriptors. The proposed method is tested on different databases consisting uncontrolled facial images such as extended Yale B, Yale A and our database. The experimental results show that the proposed method performs better at recognizing faces in uncontrolled environments such as the facial expression, illumination, and mild pose changes.

Facial expression methods play a vital role in human-computer interaction and other fields, but there are factors such as occlusion, illumination, and pose changes in wild facial recognition, as well as category imbalances between different datasets, that result in large variations in recognition rates and low accuracy rates for different categories of facial expression datasets. This study introduces RCL-Net, a method of recognizing wild facial expressions that is based on an attention mechanism and LBP feature fusion. The structure consists of two main branches, namely the ResNet-CBAM residual attention branch and the local binary feature (LBP) extraction branch (RCL-Net). First, by merging the residual network and hybrid attention mechanism, the residual attention network is presented to emphasize the local detail feature information of facial expressions; the significant characteristics of facial expressions are retrieved from both channel and spatial dimensions to build the residual attention classification model. Second, we present a locally improved residual network attention model. LBP features are introduced into the facial expression feature extraction stage in order to extract texture information on expression photographs in order to emphasize facial feature information and enhance the recognition accuracy of the model. Lastly, experimental validation is performed using the FER2013, FERPLUS, CK+, and RAF-DB datasets, and the experimental results demonstrate that the proposed method has superior generalization capability and robustness in the laboratory-controlled environment and field environment compared to the most recent experimental methods.

A linear regression based face recognition method by extending probe images

In this paper, we propose the method for pose and facial expression invariant face recognition using the affine dense SURF-like descriptors. The proposed method consists of four step, 1) we normalize the face image using the face and eye detector. 2) We apply the affine simulation for synthesizing various pose face images. 3) We make a descriptor on the overlapping block-based grid keypoints. 4) A probe image is compared with the referenced images by performing the nearest neighbor matching. To improve the recognition rate, we use the keypoint distance ratio and false matched keypoint ratio. The proposed method showed the better performance than that of the conventional methods in terms of the recognition rates.

A bi-modal face recognition framework integrating facial expression with facial appearance

Face recognition is an area of computer vision that has attracted a lot of interest from the research community. A growing demand for robust face recognition software in security applications has driven the development of interesting approaches in this field. A large quantity of research in face recognition deals with visible face images. In the visible spectrum the illumination and face expressions changes represent a significant challenge for the recognition system. To avoid these problems, researchers proposed recently the use of 3D and infrared imaging for face recognition. In this work, we introduce a new framework for infrared face recognition using texture descriptors. This framework exploits linear and non linear dimensionality reduction techniques for face learning and recognition in the texture space. Active and passive infrared imaging modalities are used and comparison with visible face recognition is performed. Two multispectral face recognition databases were used in our experiments: Equinox Database (Visible, SWIR, MWIR, LWIR) and Laval University Multispectral Database (Visible, NIR, MWIR, LWIR). The obtained results show high increase in recognition performance when texture descriptors like LBP (Local Binary Pattern) and LTP (Local Ternary Pattern) are used. The best result was obtained in the short wave infrared spectrum (SWIR) using non linear dimensionality reduction techniques.

These Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Automatic facial expression recognition system has many applications including, but not limited to, human behaviour understanding, detection of mental disorders, and synthetic human expressions. Recognition of facial expression by computer with high recognition rate is still a challenging task. Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classification. The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control

Contemporary face recognition algorithms rely on precise localization of keypoints (corner of eye, nose etc.). Unfortunately, finding keypoints reliably and accurately remains a hard problem. In this paper we pose two questions. First, is it possible to exploit the gallery image in order to find keypoints in the probe image? For instance, consider finding the left eye in the probe image. Rather than using a generic eye model, we use a model that is informed by the appearance of the eye in the gallery image. To this end we develop a probabilistic model which combines recognition and keypoint localization. Second, is it necessary to localize keypoints? Alternatively we can consider keypoint position as a hidden variable which we marginalize over in a Bayesian manner. We demonstrate that both of these innovations improve performance relative to conventional methods in both frontal and cross-pose face recognition.

Unconstrained low-resolution (LR) face recognition is still a challenging problem in computer vision. In real-world scenarios, the gallery images are generally of high-resolution (HR), while the probe images may be of low resolution. In surveillance applications, the challenge of matching LR to HR is more common because the probe images are captured in low resolution while gallery images are of high resolution. LR to HR face matching is challenging because, in the embedding space, there is a need for a common subspace for mapping the LR and HR embeddings. In LR to LR face matching, where probe and gallery both belong to low-resolution, face identification is more difficult because very less visual information is present in the images. In LR to LR face matching, the challenge becomes very hard if faces are tiny in size and belong to low-resolution. In this paper, we implement a deep learning pipeline for matching the LR to HR and LR to LR faces. Due to the absence of LR and HR images of the same identity in the real-world datasets, we have also generated the LR images from HR images using the synthetic data approach. Extensive experimental analyses have been made to compare the performance to other state-of-the-art models.

This paper develops a novel face recognition technique called Complete Gabor Fisher Classifier (CGFC). Different from existing techniques that use Gabor filters for deriving the Gabor face representation, the proposed approach does not rely solely on Gabor magnitude information but effectively uses features computed based on Gabor phase information as well. It represents one of the few successful attempts found in the literature of combining Gabor magnitude and phase information for robust face recognition. The novelty of the proposed CGFC technique comes from (1) the introduction of a Gabor phase-based face representation and (2) the combination of the recognition technique using the proposed representation with classical Gabor magnitude-based methods into a unified framework. The proposed face recognition framework is assessed in a series of face verification and identification experiments performed on the XM2VTS, Extended YaleB, FERET, and AR databases. The results of the assessment suggest that the proposed technique clearly outperforms state-of-the-art face recognition techniques from the literature and that its performance is almost unaffected by the presence of partial occlusions of the facial area, changes in facial expression, or severe illumination changes. 1.