Improving Face Recognition by Exploring Local Features with Visual Attention

In the literature of psychophysics and neurophysiology, many studies have shown that both global and local features are crucial for face representation and recognition. This paper proposes a novel face recognition method which combines both global and local discriminative features. In this method, global features are extracted from whole face images by Fourier transform and local features are extracted from some spatially partitioned image patches by Gabor wavelet transform. After this, multiple classifiers are obtained by applying Fisher Discriminant Analysis on global Fourier features and local patches of Gabor features. All these classifiers are combined to form a hierarchical ensemble by sum rule. We evaluated the proposed method using Face Recognition Grand Challenge (FRGC) experimental protocols and database known as the largest data sets available. Experimental results on FRGC version 2.0 data set have shown that the proposed method achieves a verification rate of 86%, while the best reported was 76%.

LocalFace: Learning significant local features for deep face recognition

One of the main issues of face recognition is to extract features from face images, which include both local and global features. We present a novel method to perform feature fusion at the feature level. First, global features are extracted by principal component analysis (PCA), while local features are obtained by active appearance model (AAM) and Gabor wavelet transform (GWT). Second, two types of features are fused by weighted concatenation. Finally, Euclidean and feature distances of fused features are applied to carry out a nearest neighbor classifier. The method is evaluated by the recognition rates and computation cost over two face image databases [AR (created by A. Martinez and R. Benavente) and SJTU-IPPR (Shanghai JiaoTong University-Institute of Image Processing and Pattern Recognition)]. Compared with PCA and elastic bunch graph matching (EBGM), the presented method is more effective. Though the recognition rate of the presented method is not as good as nonlinear feature combination (NFC), low computation cost is its superiority. In addition, experimental results show that the novel method is robust to variations over time, expression, illumination, and pose to a certain extent.

Summary form only give. Face recognition can be defined as the ability of a system to classify or describe a human face. The motivation for such system is to enable computers to do things like humans do and to apply computers to solve problems that involve analysis and classification. Face recognition systems require less user cooperation than systems based on other biometrics (e.g. fingerprints and iris), it is one of the most widely investigated biometric techniques for human identification and it can be used in applications such as access control, passport control, surveillance, criminal justice and human computer interaction. Face recognition is a specific case of object recognition. It is not a unique and rigid object. Indeed, Global features are sensitive to variations caused by emotional expressions, illumination, pose and occlusions. Neural networks have been widely used for applications related to face recognition and Backpropagation Neural Network (BPNN) is one of the most widely used methods in this domain. In this paper we present 3 solutions related to neural network for color face recognition. First we introduce learning-based dimension reduction algorithms. In the literature many methods are used to reduce the dimensionality of the subspace in which faces are presented. Recently, Random Projection (RP) has emerged as a powerful method for dimensionality reduction. It represents a computationally simple and efficient method that preserves the structure of the data without introducing very significant distortion. Our focus was to investigate the dimensionality reduction offered by RP and perform an artificial intelligent system for face recognition. According to the experimental results, we conclude that random projection is an optimal method of dimensionality reduction. In the case of our study, obtaining a higher FR rate depends, among others, on the choice of the random projection matrix and the dimension of the feature vector of original data. Secondly, we propose a hybrid method to achieve face recognition purpose using semi supervised BPNN. Traditionally, BPNN needs supervised training to learn how to predict results from desired data, the idea of our approach is to get the desired output of the network from an exterior classifier (SOM) and then apply the back propagation algorithm to recognize facial data. Experiments show that the results are satisfying in comparison with the supervised BPNN. Furthermore, we can deduce that the unlabeled vector in the training DB generally does not influence the recognition task and due to its generation ability the neural net can even correct some misclassified vectors. The third study concerns the use of Bhattacharyya distance to calculate the total error of the network. The error function generally used to train the neural network is Mean Square Error (MSE) based on Euclidean distance measure. In the experimental section we compare how the algorithm converge using the Mean Square Error and the Bhataccharyya distance and results indicated that the image faces can be recognized by the proposed system effectively and swiftly.

This study analyzes the effectiveness of the global (the whole face) and local (regions of eyes, nose, and mouth) features for face recognition. Features describing human faces are encoded in local ternary patterns. The two-class support vector machine is used as the supervised learning algorithm for training recognition models. In the recognition process, recognition modes based on the global features and local features are cascaded. For identifying a face image, the local features are used iteratively for filtering out candidates that can not be clearly identified by the global features, until the one with highest possibility is concluded. The experimental results show that cascading the recognition models of global and local features obtains better classification accuracy than the single classification process.

Human face recognition is one of the most important research areas in biometrics. However, the robust face recognition under a drastic change of the facial pose, expression, and illumination is a big challenging problem for its practical application. Such variations make face recognition more difficult. In this paper, we propose a novel face recognition method, called Attentional Feature-pair Relation Network (AFRN), which represents the face by the relevant pairs of local appearance block features with their attention scores. The AFRN represents the face by all possible pairs of the 9x9 local appearance block features, the importance of each pair is considered by the attention map that is obtained from the low-rank bilinear pooling, and each pair is weighted by its corresponding attention score. To increase the accuracy, we select top-K pairs of local appearance block features as relevant facial information and drop the remaining irrelevant. The weighted top-K pairs are propagated to extract the joint feature-pair relation by using bilinear attention network. In experiments, we show the effectiveness of the proposed AFRN and achieve the outstanding performance in the 1:1 face verification and 1:N face identification tasks compared to existing state-of-the-art methods on the challenging LFW, YTF, CALFW, CPLFW, CFP, AgeDB, IJB-A, IJB-B, and IJB-C datasets.

We investigated the application of multiresolution global and local texture features to reduce false-positive detection in a computerized mass detection program. One hundred and sixty-eight digitized mammograms were randomly and equally divided into training and test groups. From these mammograms, two datasets were formed. The first dataset (manual) contained four regions of interest (ROIs) selected manually from each of the mammograms. One of the four ROIs contained a biopsy-proven mass and the other three contained normal parenchyma, including dense, mixed dense/fatty, and fatty tissues. The second dataset (hybrid) contained the manually extracted mass ROIs, along with normal tissue ROIs extracted by an automated Density-Weighted Contrast Enhancement (DWCE) algorithm as false-positive detections. A wavelet transform was used to decompose an ROI into several scales. Global texture features were derived from the low-pass coefficients in the wavelet transformed images. Local texture features were calculated from the suspicious object and the peripheral subregions. Linear discriminant models using effective features selected from the global, local, or combined feature spaces were established to maximize the separation between masses and normal tissue. Receiver Operating Characteristic (ROC) analysis was conducted to evaluate the classifier performance. The classification accuracy using global features were comparable to that using local features. With both global and local features, the average area, Az, under the test ROC curve, reached 0.92 for the manual dataset and 0.96 for the hybrid dataset, demonstrating statistically significant improvement over those obtained with global or local features alone. The results indicated the effectiveness of the combined global and local features in the classification of masses and normal tissue for false-positive reduction.

Facial feature selection is one of the most important challenges in a face recognition system. In a face image, only a part of a face image is changed due to illumination, pose, and other source of changes. In this paper, a novel face recognition approach is proposed based on fusing global and local features. To extract global and local features, we employed Gabor wavelet filter to apply on whole image and non-overlapping sub-images with equal size. To reduce the dimension of new fused feature vector, Principal Component Analysis (PCA) technique is employed. In our experiments, we used KNN and multi-class SVM classifiers and ORL database to obtain face recognition rate. The results show that the new face recognition algorithm outperforms the conventional methods such as global Gabor face recognition, and G-2DFLD feature fusion face recognition in term of recognition rate.

In this study, we investigated how shapes are classified based on local and global features by four representative convolutional neural networks (CNNs), i.e., AlexNet, VGG, ResNet and Inception. While the local features are based on simple components, such as orientation of line segment, the global features are based on the whole object, such as whether an object has a hole. For example, solid triangles and solid squares are differentiated by local features, solid circles and rings are differentiated by a global feature. Two sets of experiments were performed in this research. In the first experiment, we examined how the four CNNs pre-trained on ImageNet (with transfer learning) learned to differentiate the regular shapes (equilateral-triangles, squares, circles and rings). Our results showed that the pre-trained CNNs exhibited faster learning rates in the tasks discriminating the local features than in the tasks discriminating the global feature. However, the transfer learning of discriminating the global feature in regular shapes were better generalized to irregular shapes than the transfer learning of discriminating local features. In the second experiment, the CNNs were trained from scratch (with random weights initialization) to discriminate local and global features in regular and irregular shapes. Different from the transfer learning, the CNNs exhibited faster learning rates in discriminating the global feature than the local features. Similar to transfer learning, the CNNs exhibited excellent learning generalization to discriminating the global feature of irregular shapes, but poor learning generalization to discriminating the local features in the irregular shapes. The overarching goal of this research is to create a paradigm and benchmark to directly compare how the CNNs and primate visual systems process geometrical invariants. In contrast to the ImageNet approach which employs natural images to train CNNs, we employed the “ShapeNet” approach which features geometrical shapes with well-defined properties. The ShapeNet approach will not only help elucidate the strengths and limitations of CNN computation, but also provide insights into visual information processing in the primates.

Face recognition is considered as one of the relatively new and interesting concepts in the area of biometrics and comprises a huge number of applications. This study involves implementation of a robust recognition system by employing global and random local facial features of an individual. The proposed scheme considers the extraction of global facial features and some randomly selected local facial features. In order to exploit global information of a whole face image, Principal Component Analysis (PCA) algorithm is used. On the other hand, the randomly selected sub-images of a whole face image are concatenated and subsequently PCA is applied on the concatenated regions. In addition, the proposed scheme involves the optimization of random sub-image locations and sizes by considering different settings and choosing the most optimized one. Finally, Weighted Sum Rule fusion is employed to combine the calculated scores of the global and local feature extractors. The reliability of the proposed facial recognition system is investigated on several data sets of ORL, FERET, and Extended Yale face databases. Demonstration of results based on the recognition performance and ROC analysis clarifies that the proposed scheme achieves a considerable improvement compared to the global and local feature extractors implemented in this study.

Sentence similarity is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. However, a variety of linguistic expressions and ambiguities of words in sentences make it difficult to measure sentence similarity. Many studies show that using local features or global features of a sentence will produce satisfactory sentence representations that can be utilized to measure sentence similarity. Local features reflect the relationships of adjacent words for each sentence and the sequence information of a sentence are usually expressed by global features. However, local features lack abilities to capture sequence information while a small amount of extracted global features is not enough to produce sentence representations with good qualities. In this paper, we propose A Hybrid Model combining Local and Global features into a Siamese Network (HM-LGSN) for sentence similarity calculation. We first propose a new convolution neural network architecture called group convolution neural network to extract the most representative local features (or word semantic features). Then we combine these new features with pre-trained embeddings of words as input to the Bidirectional Gated Recurrent Units to extract global features of sentences. Finally, we select the global features to form sentence representations and calculate sentence similarity through Manhattan distance. The experimental results on SICK, MSRVID, STS-B datasets show that the accuracy of our proposed model is significantly improved by combining local features and global features.

One important type of biometric authentication is face recognition , a research area of high popularity with a wide spectrum of approaches that have been proposed in the last few decades. The majority of existing approaches are conceived for or evaluated on constrained still images. However, more recently research interests have shifted toward unconstrained “in-the-wild ” still images and videos. To some extent, current state-of-the-art systems are able to cope with variability due to pose, illumination, expression, and size, which represent the challenges in unconstrained face recognition. To date, only few attempts have addressed the problem of face recognition in mobile environment , where high degradation is present during both data acquisition and transmission. This book chapter deals with face recognition in mobile and other challenging environments, where both still images and video sequences are examined. We provide an experimental study of one commercial off-the-shelf (COTS) and four recent open-source face recognition algorithms , including color-based linear discriminant analysis (LDA) , local Gabor binary pattern histogram sequences (LGBPHSs) , Gabor grid graphs , and intersession variability (ISV) modeling . Experiments are performed on several freely available challenging still image and video face databases, including one mobile database, always following the evaluation protocols that are attached to the databases. Finally, we supply an easily extensible open-source toolbox to rerun all the experiments, which includes the modeling techniques, the evaluation protocols, and the metrics used in the experiments and provides a detailed description on how to regenerate the results.

Many face recognition systems have demonstrated promising results under well-controlled conditions with cooperative users. However, face recognition in real-world scenarios is still a challenging problem due to dramatic facial variations caused by different poses, lighting conditions, expressions, occlusion and so on. In this chapter, we summarize recent advances in unconstrained face recognition. We begin by introducing existing unconstrained face databases or benchmarks. We then provide an overview of recent techniques specifically developed for this task, including advanced face representations, metric learning approaches, background information investigation and pose-invariant approaches. Finally, we highlight some open issues to be addressed.

In view of the fact that information which only contained in global or local features is insufficient, this paper proposes a novel face recognition method based on Hybrid Binary Pattern (HBP). In this paper, we expect better performance of face recognition by combining global and local features. On the one hand, we introduce a new rotation invariant texture representation method to extract the BGP feature of the face image. On the other hand, we extract the Multi-scale LBP (MSLBP) feature by using Wavelet Transform (WT) and LBP. Finally, we can combine the BGP feature and MSLBP feature to form a HBP feature. In the face recognition process, we adopt the Nearest Neighbor (NN) classifier for its simplicity. The experiment results show that the proposed method integrates different features efficiently and has better robustness to variations of lighting, expression and pose.

Human perception of the face involves the observation of both coarse (global) and detailed (local) features of the face to identify and categorize a person. Face categorization involves finding common visual cues, such as gender, race and age, which could be used as a precursor to a face recognition system to improve recognition rates. In this paper, we investigate the fusion of both global and local features for gender classification. Global features are obtained using the principal component analysis (PCA) and discrete cosine transformation (DCT) approaches. A spatial local binary pattern (LBP) approach augmented with a two-dimensional DCT approach has been used to find the local features. The performance of the proposed approach has been investigated through extensive experiments performed on FERET database. The proposed approach gives a recognition accuracy of 98.16% on FERET database. Comparisons with some of the existing techniques have shown a marked reduction in number of features used per image to produce results more efficiently and without loss of accuracy for gender classification.