Attentional Feature-Pair Relation Networks for Accurate Face Recognition

In the literature of psychophysics and neurophysiology, many studies have shown that both global and local features are crucial for face representation and recognition. This paper proposes a novel face recognition method which combines both global and local discriminative features. In this method, global features are extracted from whole face images by Fourier transform and local features are extracted from some spatially partitioned image patches by Gabor wavelet transform. After this, multiple classifiers are obtained by applying Fisher Discriminant Analysis on global Fourier features and local patches of Gabor features. All these classifiers are combined to form a hierarchical ensemble by sum rule. We evaluated the proposed method using Face Recognition Grand Challenge (FRGC) experimental protocols and database known as the largest data sets available. Experimental results on FRGC version 2.0 data set have shown that the proposed method achieves a verification rate of 86%, while the best reported was 76%.

We propose a novel face recognition method using a Hierarchical Feature Relational Network (HFRN) which extracts facial part representations around facial landmark points, and predicts hierarchical latent relations between facial part representations. These hierarchical latent relations should be unique relations within the same identity and discriminative relations among different identities for face recognition task. To do this, the HFRN extracts appearance features as facial parts representations around facial landmark points on the feature maps, globally pool these extracted appearance features onto single feature vectors, and captures the relations for the pairs of appearance features. The HFRN captures the locally detailed relations in the low-level layers and the locally abstracted global relations in the high-level layers for the pairs of appearance features extracted around facial landmark points projected on each layer, respectively. These relations from low-level layers to high-level layers are concatenated into a single hierarchical relation feature. To further improve the accuracy of face recognition, we combine the global appearance feature with the hierarchical relation feature. In experiments, the proposed method achieves the comparable performance in the 1:1 face verification and 1:N face identification tasks compared to existing state-of-the-art methods on the challenging IARPA Janus Benchmark A (IJB-A) and IARPA Janus Benchmark B (IJB-B) datasets.

In this paper, we present an extensive evaluation of face recognition and verification approaches performed by the European COST Action MULTI-modal Imaging of FOREnsic SciEnce Evidence (MULTI-FORESEE). The aim of the study is to evaluate various face recognition and verification methods, ranging from methods based on facial landmarks to state-of-the-art off-the-shelf pre-trained Convolutional Neural Networks (CNN), as well as CNN models directly trained for the task at hand. To fulfill this objective, we carefully designed and implemented a realistic data acquisition process, that corresponds to a typical face verification setup, and collected a challenging dataset to evaluate the real world performance of the aforementioned methods. Apart from verifying the effectiveness of deep learning approaches in a specific scenario, several important limitations are identified and discussed through the paper, providing valuable insight for future research directions in the field.

In this paper, we propose a novel face recognition method based on local appearance feature extraction using dual-tree complex wavelet transform (DT-CWT). It provides a local multiscale description of images with good directional selectivity and invariance to shifts and in-plane rotations. In the dual-tree implementation, two parallel discrete wavelet transform (DWT) with different lowpass and highpass filters in different scales were used. The linear combination of subbands generated by two parallel DWT is used to generate 6 different directional subbands with complex coefficients. It is insensitive to illumination variations and facial expression changes. 2-D dual-tree complex wavelet transform is less redundant and computationally efficient. The local DT-CWT coefficients are used to extract the facial features which improve the face recognition with small sample size in less computation. The local features based methods have been successfully applied to face recognition and achieved state-of-the-art performance. Normally most of the local appearance based methods the facial features are extracted from several local regions and concatenated into an enhanced feature vector as a face descriptor. In this approach we divide the face into several (m ? m) non-overlapped parallelogram blocks instead of square or rectangle blocks. The local mean, standard deviation and energy of complex wavelet coefficients are used to describe the face image. Experiments, on two well-known databases, namely, Yale and ORL databases, shows the Local DT-CWT approach performs well on illumination, expression and perspective variant faces with single sample compared to PCA and global DT-CWT. Furthermore, in addition to the consistent and promising classification performances, our proposed Local DT-CWT based method has a really low computational complexity.

Local fusion of complex dual-tree wavelet coefficients based face recognition for single sample problem

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)-adapted from PCA-SIFT-is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose.

The automatic recognition of human faces presents a significant challenge to the pattern recognition research community. Typically, human faces are very similar in structure with minor differences from person to person. They are actually within one class of "human face". Furthermore, lighting conditions change, while facial expressions and pose variations further complicate the face recognition task as one of the difficult problems in pattern analysis. This paper proposes a novel concept: namely, that faces can be recognized using a line edge map (LEM). The LEM, a compact face feature, is generated for face coding and recognition. A thorough investigation of the proposed concept is conducted which covers all aspects of human face recognition, i.e. face recognition under (1) controlled/ideal conditions and size variations, (2) varying lighting conditions, (3) varying facial expressions, and (4) varying pose. The system performance is also compared with the eigenface method, one of the best face recognition techniques, and with reported experimental results of other methods. A face pre-filtering technique is proposed to speed up the search process. It is a very encouraging to find that the proposed face recognition technique has performed better than the eigenface method in most of the comparison experiments. This research demonstrates that the LEM, together with the proposed generic line-segment Hausdorff distance measure, provides a new method for face coding and recognition.

Most face recognition methods employ single-bit binary descriptors for face representation. The information from these methods is lost in the process of quantization from real-valued descriptors to binary descriptors, which greatly limits their robustness for face recognition. In this study, we propose a novel weighted feature histogram (WFH) method of multi-scale local patches using multi-bit binary descriptors for face recognition. First, to obtain multi-scale information of the face image, the local patches are extracted using a multi-scale local patch generation (MSLPG) method. Second, with the goal of reducing the quantization information loss of binary descriptors, a novel multi-bit local binary descriptor learning (MBLBDL) method is proposed to extract multi-bit local binary descriptors (MBLBDs). In MBLBDL, a learned mapping matrix and novel multi-bit coding rules are employed to project pixel difference vectors (PDVs) into the MBLBDs in each local patch. Finally, a novel robust weight learning (RWL) method is proposed to learn a set of robust weights for each patch to integrate the MBLBDs into the final face representation. In RWL, a codebook is first constructed by clustering MBLBDs on each local patch to extract a feature histogram. Then, considering that different parts of the face have different degrees of robustness to local changes, a set of weights is learned to concatenate the feature histograms of all local patches into the final representation of a face image. In addition, to further improve the performance for heterogeneous face recognition, a coupled WFH (C-WFH) method is proposed. C-WFH maintains the similarity of the corresponding MBLBDs and feature histograms for a pair of heterogeneous face images by means of a novel coupled feature learning (CFL) method to reduce the modality gap. A series of experiments are conducted on widely used face datasets to analyze the performance of WFH and C-WFH. Extensive experimental results show that WFH and C-WFH outperform state-of-the-art face recognition methods.

Lack of training samples always affects the performance and robustness of face recognition. Generating virtual samples is one of effective methods to expand the training set. When the virtual samples are able to simulate the variations of facial images including variations of illuminations, facial postures and the facial expressions, the robustness will be enhanced and the accuracy will be improved obviously in the face recognition problem. In this paper, an improved linear representation-based classification combined virtual samples (ILRCVS) is proposed. First, we design a new objective function that simultaneously considers the information of the virtual training samples and the virtual test sample. Second, an alternating minimization algorithm is proposed to solve the optimization problem of the objective function. Finally, a new classification criterion combined with the virtual training and test sample is proposed. Experimental results on the Georgia Tech, FERET and Yale B face databases show that the proposed method is more robust than three state-of-the-art face recognition methods, LRC, SRC and CRC.

The equivalence of two-dimensional PCA to line-based PCA

LocalFace: Learning significant local features for deep face recognition

Facial albedo images are believed to be invariant to external factors of pose, illumination and expression that can greatly affect the appearance of face images and thus face recognition accuracy as well. Unlike most existing face recognition methods that address the impact of one or two of these external factors, we propose an end-to-end network, which consists of De-Light Network (DL-Net) and Normalization Network (N-Net), to generate normalized albedo images with neutral expression and frontal pose for input face images. DL-Net aims to eliminate the effects of illumination and reconstruct a posed albedo image that has the same pose and expression as the input image. N-Net attempts to generate a pose and expression normalized albedo image and extract identity features under the supervision of the normalized albedo images. Our experiments on the Multi-PIE database show that the extracted identity features can effectively assist conventional face recognition methods to improve face recognition accuracy under varying poses, illuminations and expressions.

For the past years, studies on face recognition were focused on the effects and influence of illumination, pose, and angle to the performance, which, in fairness, has been proven by different research works as the dominant factor of adding complexity to face recognition across age progression observed through photographs, specifically headshots (Ling, Soatto, Ramanathan, & Jacobs, 2007; Ramanathan & Chellappa, 2006; Ramanathan & Chellappa, 2005). However, studies on face verification and recognition with time lapses between pairs of photographs and the interaction on the sexual orientation of the identifier was somehow defocused on the factor of effects of age progression or the time interval between pairs of facial images on face recognition. Thus, the researchers aimed to resolve the gap on the prevailing literatures on face recognition by finding the effects of age interval of the stimuli and sexual orientation of the participants on face recognition using Filipino college students as sample. Results showed that the greater the age gaps between the pair of images, the lesser the chance of identifying it. In addition, it is also not important to consider whether identifier is male or female in terms of recognizing picture because based from the result of the experiment, sexual orientation does not have an effect on the performance of the participants in face recognition. The study was guided by the theory of Biederman (1987) and Recognition By Components (RBC) to analyze the findings.

We propose a novel face recognition method that represents and classifies face images in the feature space. It first assumes that in the feature space the test sample can be well expressed by a linear combination of the training samples, and then it exploits the obtained linear combination to perform face recognition. We also present the foundation, rationale, and characteristics of, as well as the differences between, our method and conventional kernel methods. The analysis shows that our method is a representation-based kernel method and works in the feature space. This method might be able to outperform the representation-based methods that work in the original space. The experimental results show that our method partially possesses the properties of "sparseness" and is able to reduce greatly the effects of noise and occlusion in the test sample.

Over the past several years, the performance of state-of-the-art face recognition systems has been significantly improved, due in a large part to the increasing amount of available face datasets and the proliferation of deep neural networks. This rapid increase in performance has left existing popular performance evaluation protocols, such as standard LFW, nearly saturated and has motivated the emergence of new, more challenging protocols (aimed specifically towards unconstrained face recognition). In this work, we employ the use of parts-based face recognition models to further improve the performance of state-of-the-art face recognition systems as evaluated by both the LFW protocol, and the newer, more challenging protocols (BLUFR, IJB-A, and IJB-B). In particular, we employ spatial transformers to automatically localize discriminative facial parts which enables us to build an end-to-end network where global features and local features are fused together, making the final feature representation more discriminative. Experimental results, using these discriminative features, on the BLUFR, IJB-A and IJB-B protocols, show that the proposed approach is able to boost performance of state-of-the-art face recognition systems. The pro-posed approach is not limited to one architecture but can also be applied to other face recognition networks.