The Complete Gabor-Fisher Classifier for Robust Face Recognition

Face recognition is an area of computer vision that has attracted a lot of interest from the research community. A growing demand for robust face recognition software in security applications has driven the development of interesting approaches in this field. A large quantity of research in face recognition deals with visible face images. In the visible spectrum the illumination and face expressions changes represent a significant challenge for the recognition system. To avoid these problems, researchers proposed recently the use of 3D and infrared imaging for face recognition. In this work, we introduce a new framework for infrared face recognition using texture descriptors. This framework exploits linear and non linear dimensionality reduction techniques for face learning and recognition in the texture space. Active and passive infrared imaging modalities are used and comparison with visible face recognition is performed. Two multispectral face recognition databases were used in our experiments: Equinox Database (Visible, SWIR, MWIR, LWIR) and Laval University Multispectral Database (Visible, NIR, MWIR, LWIR). The obtained results show high increase in recognition performance when texture descriptors like LBP (Local Binary Pattern) and LTP (Local Ternary Pattern) are used. The best result was obtained in the short wave infrared spectrum (SWIR) using non linear dimensionality reduction techniques.

The paper introduces a feature extraction technique for face recognition called the Phase-based Gabor Fisher Classifier (PBGFC). The PBGFC method constructs an augmented feature vector which encompasses Gabor-phase information derived from a novel representation of face images - the oriented Gabor phase congruency image (OGPCI) - and then applies linear discriminant analysis to the augmented feature vector to reduce its dimensionality. The feasibility of the proposed method was assessed in a series of face verification experiments performed on the XM2VTS database. The experimental results show that the PBGFC method performs better than other popular feature extraction techniques such as principal component analysis (PCA), the Fisherface method or the DCT-mod2 approach, while it ensures similar verification performance as the established Gabor Fisher Classifier (GFC). The results also show that the proposed Phase-based Gabor Fisher Classifier performs the best among the tested methods when severe illumination changes are introduced to the face images.

A bi-modal face recognition framework integrating facial expression with facial appearance

In the current scenario face identification and recognition is an important technique in surveillance. The face is a necessary biometric in humans. Therefore face detection plays a major job in computer vision applications. Several face recognition and emotions classification approaches have been presented throughout the last few decades of research to improve the rate of face recognition for thermal pictures. However, in real-time, lighting conditions might change due to several factors, such as the different times of capture, weather, etc. Due to variations in lighting intensity, the performance of the facial expression recognition system is not good. This paper proposed a model for human thermal face detection and expression classification. Four main steps were involved in this research. Initially, the Difference of the Gaussian (DOG) filter is utilized to crop the input thermal images and then normalize the images using the median filter in pre-processing step. Then, Efficient Net is used for extracting features such as shape, location, and occurrences from thermal face images. After that, detect human faces utilized by the YOLOv4 technique to better emotions classification. Finally, classify the emotions on faces by using the DenseNet technique into seven emotions such as happy, sad, disgust, surprise, anger, fear, and neutral. The proposed method outperforms state-of-art techniques for face recognition on thermal pictures, and classifies the expressions, according to experimentations on the RGB-D-T database. The accuracy, precision, recall, and f1-score metrics will be utilized with the database to assess the efficacy of the proposed methodology. The proposed models achieve a high classification accuracy of 95.97% on the RGB-D-T database. Furthermore, the outcomes show good precision for various face recognition tasks.

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods.

The automatic recognition of human faces presents a significant challenge to the pattern recognition research community. Typically, human faces are very similar in structure with minor differences from person to person. They are actually within one class of "human face". Furthermore, lighting conditions change, while facial expressions and pose variations further complicate the face recognition task as one of the difficult problems in pattern analysis. This paper proposes a novel concept: namely, that faces can be recognized using a line edge map (LEM). The LEM, a compact face feature, is generated for face coding and recognition. A thorough investigation of the proposed concept is conducted which covers all aspects of human face recognition, i.e. face recognition under (1) controlled/ideal conditions and size variations, (2) varying lighting conditions, (3) varying facial expressions, and (4) varying pose. The system performance is also compared with the eigenface method, one of the best face recognition techniques, and with reported experimental results of other methods. A face pre-filtering technique is proposed to speed up the search process. It is a very encouraging to find that the proposed face recognition technique has performed better than the eigenface method in most of the comparison experiments. This research demonstrates that the LEM, together with the proposed generic line-segment Hausdorff distance measure, provides a new method for face coding and recognition.

This paper proposes a 2D face recognition algorithm using phase-based correspondence matching. The phase information obtained from 2D DFT (Discrete Fourier Transform) of images contains important information of image representation. The phase-based image matching is successfully applied to sub-pixel image registration tasks for computer vision applications and image recognition tasks for biometric authentication applications. Hierarchical block matching using phase information, i.e, phase-based correspondence matching, can find the corresponding points on the input image from the reference points on the registered image with sub-pixel accuracy. For face recognition, the phase-based correspondence matching is useful for minute change of texture, such as facial expression change, illumination change, etc. Experimental evaluation using the CSU Face Identification Evaluation System with the FERET database demonstrates efficient recognition performance of the proposed algorithm compared with the conventional face recognition algorithms.

Learning affective representations based on magnitude and dynamic relative phase information for speech emotion recognition

This paper details the results of a face verification competition [2] held in conjunction with the Second International Conference on Biometric Authentication. The contest was held on the publically available XM2VTS database [4] according to a defined protocol [15]. The aim of the competition was to assess the advances made in face recognition since 2003 and to measure the sensitivity of the tested algorithms to severe changes in illumination conditions. In total, more than 10 algorithms submitted by three groups were compared. The results show that the relative performance of some algorithms is dependent on training conditions (data, protocol) as well as environmental changes.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

Current automatic facial recognition systems are not robust against changes in illumination, pose, facial expression and occlusion. In this paper, we propose an algorithm based on a probabilistic approach for face recognition to address the problem of pose change by a probabilistic approach that takes into account the pose difference between probe and gallery images. By using a large facial image database called CMU PIE database, which contains images of the same set of people taken from many different angles, we have developed a probabilistic model of how facial features change as the pose changes. This model enables us to make our face recognition system more robust to the change of poses in the probe image. The experimental results show that this approach achieves a better recognition rate than conventional face recognition methods over a much larger range of pose. For example, when the gallery contains only images of a frontal face and the probe image varies its pose orientation, the recognition rate remains within a less than 10% difference until the probe pose begins to differ more than 45 degrees, whereas the recognition rate of a PCA-based method begins to drop at a difference as small as 10 degrees, and a representative commercial system at 30 degrees.

2D face recognition based on supervised subspace learning from 3D models

Facial expressions are undoubtedly the most effective nonverbal communication. The face has always been the equation of a person's identity. The face draws the demarcation line between identity and extinction. Each line on the face adds an attribute to the identity. These lines become prominent when we experience an emotion and these lines do not change completely with age. In this paper we have proposed a new technique for face recognition which focuses on the facial expressions of the subject to identify his face. This is a grey area on which not much light has been thrown earlier. According to earlier researches it is difficult to alter the natural expression. So our technique will be beneficial for identifying occluded or intentionally disguised faces. The test results of the experiments conducted prove that this technique will give a new direction in the field of face recognition. This technique will provide a strong base to the area of face recognition and will be used as the core method for critical defense security related issues.

After decades of extensive research, the field of 2D face recognition is fruitful. However, 2D face recognition is sensitive to changes in pose, facial expression and illumination. 3D face recognition has great potential due to the inherent invariance of pose and illumination changes, but these 3D face images are still difficult to acquire due to several issues such as cost and accessibility. In contrast, RGB-D images captured by low-cost depth sensors such as Kinect are relatively easy to acquire, and research on such low-quality inputs is still limited. In this thesis, an end-to-end multimodal fusion method based on spatial and channel attention is proposed to effectively fuse two image modalities, RGB and depth, to enhance RGB-D face recognition. Our work is specified as follows: firstly, three branches combined with the spatial and channel attention modules are established on the basis of ResNet18, so that the features of three modalities (RGB, depth map, and their fusion modalities, respectively) are obtained. The features under these three modalities are then fused, fed into a shared layer, and the fused features are further learned for deeper discriminative features and passed through a spatial attention vectorization module at the end. The method achieves good results on the Intellifusion RGBD dataset, while we perform ablation trials, which well reveal the contribution of each component of our method to the final performance.

A novel feature selection strategy for improved face recognition in images with variations due to illumination conditions, facial expressions, and partial occlusions is presented in this dissertation. A hybrid face recognition system that uses feature maps of phase congruency and modular kernel spaces is developed. Phase congruency provides a measure that is independent of the overall magnitude of a signal, making it invariant to variations in image illumination and contrast. A novel modular kernel spaces approach is developed and implemented on the phase congruency feature maps. Smaller sub-regions from a predefined neighborhood within the phase congruency images of the training samples are merged to obtain a large set of features. These features are then projected into higher dimensional spaces using kernel methods. The unique modularization procedure developed in this research takes into consideration that the facial variations in a real world scenario are confined to local regions. The additional pixel dependencies that are considered based on their importance help in providing additional information for classification. This procedure also helps in robust localization of the variations, further improving classification accuracy. The effectiveness of the new feature selection strategy has been demonstrated by employing it in two specific applications via face authentication in low resolution cameras and face recognition using multiple sensors (visible and infrared). The face authentication system uses low quality images captured by a web camera. The optical sensor of the web camera is very sensitive to environmental illumination variations. It is observed that the feature selection policy overcomes the facial and environmental variations. A methodology based on multiple training images and clustering is also incorporated to overcome the additional challenges of computational efficiency and the subject's non involvement. A multi-sensor image fusion based face recognition methodology that uses the proposed feature selection technique is presented in this dissertation. Research studies have indicated that complementary information from different sensors helps in improving the recognition accuracy compared to individual modalities. A decision level fusion methodology is also developed which provides better performance compared to individual as well as data level fusion modalities. The new decision level fusion technique is also robust to registration discrepancies, which is a very important factor in operational scenarios. Research work is progressing to use the new face recognition technique in multi-view images by employing independent systems for separate views and integrating the results with an appropriate voting procedure.