Adaptive convolution local and global learning for class-level joint representation of face recognition with single sample per person

Joint and collaborative representation with local adaptive convolution feature for face recognition with single sample per person

Multi-level dynamic error coding for face recognition with a contaminated single sample per person

Real-world face recognition systems often have to face the single sample per person (SSPP) problem, that is, only a single training sample for each person is enrolled in the database. In this case, many of the popular face recognition methods fail to work well due to the inability to learn the discriminatory information specific to the persons to be identified. To address this problem, in this paper, we propose an Adaptive Generic Learning (AGL) method, which adapts a generic discriminant model to better distinguish the persons with single face sample. As a specific implementation of the AGL, a Coupled Linear Representation (CLR) algorithm is proposed to infer, based on the generic training set, the within-class scatter matrix and the class mean of each person given its single enrolled sample. Thus, the traditional Fisher's Linear Discriminant (FLD) can be applied to SSPP task. Experiments on the FERET and a challenging passport face database show that the proposed method can achieve better results compared with other common solutions to the SSPP problem.

Face recognition (FR) with a single sample per person (SSPP) is one of the most challenging problems in computer vision. In this scenario, it is difficult to predict facial variation such as pose, illumination, and disguise due to the lack of enough training samples. Therefore, the development of the FR system with only a small number of training samples is hindered. To address this problem, this paper proposes a scheme combined transfer learning and sample expansion in feature space. First, it uses transfer learning by training a deep convolutional neural network on a common multi-sample face dataset and then applies the well-trained model to a target data set. Second, it proposes a sample expansion method in feature space called k class feature transfer (KCFT) to enrich intra-class variation information for a single-sample face feature. Compared with other expanding sample methods in the image domain, this method of expanding the samples in the feature domain is novel and easy to implement. Third, it trains a softmax classifier with expanded face features. The experimental results on ORL, FERET, and LFW face databases demonstrate the effectiveness and robustness of the proposed method for various facial variations.

Face Recognition (FR) problem is one of the significant fields in computer vision. FR is used to identify the faces that appear over distributed cameras over the network. The problem of face recognition can be divided into two categories, the first is recognition with more than one sample per person, which can be called traditional face recognition problem. The second is the recognition of faces using only a Single Sample Per Person (SSPP). The efficiency of face recognition systems decreases because of limited references especially (SSPP) and faces taken in the Operational Domain (OD) different from faces in the Enrollment Domain (ED) in illumination, pose, low-resolution, and blurriness. This paper proposed a method that deals with all problems related to face recognition with SSPP. 3D face reconstruction is used to increase the reference gallery set with different poses and generate a design domain dictionary to overcome the problem of limited reference. Besides, the design domain dictionary is used to feed different deep learning models. Face illumination transfer techniques are utilized to overcome the illumination problem. Labeled Faces in the Wild (LFW) dataset is used to train Super-Resolution Generative Adversarial Network (SRGAN) to overcome the low-resolution problem. Deblur Generative Adversarial Network (DeblurGAN) is trained on the LFW dataset to overcome the problem of blurriness. The proposed method is evaluated using the Chokepoint dataset and COX-S2V dataset. The final results confirm an overall enhancement in accuracy compared to techniques that use SSPP for face recognition (generic learning and face synthesizing approaches). Also, the proposed method outperforms of Traditional and Deep Learning (TDL) method accuracy, which uses SSPP for face recognition.

Constrained [formula omitted]-optimal sparse representation technique for face recognition

The issue of single sample per person (SSPP) face recognition has attracted more and more attention in recent years. Patch/local-based algorithm is one of the most popular categories to address the issue, as patch/local features are robust to face image variations. However, the global discriminative information is ignored in patch/local-based algorithm, which is crucial to recognize the nondiscriminative region of face images. To make the best of the advantage of both local information and global information, a novel two-layer local-to-global feature learning framework is proposed to address SSPP face recognition. In the first layer, the objective-oriented local features are learned by a patch-based fuzzy rough set feature selection strategy. The obtained local features are not only robust to the image variations, but also usable to preserve the discrimination ability of original patches. Global structural information is extracted from local features by a sparse autoencoder in the second layer, which reduces the negative effect of nondiscriminative regions. Besides, the proposed framework is a shallow network, which avoids the over-fitting caused by using multilayer network to address SSPP problem. The experimental results have shown that the proposed local-to-global feature learning framework can achieve superior performance than other state-of-the-art feature learning algorithms for SSPP face recognition.

Surveillance video face recognition with single sample per person based on 3D modeling and blurring

Cycle-autoencoder based block-sparse joint representation for single sample face recognition

This paper proposes a novel method for recognizing facial images based on the relative distances between an input image and example images. Example facial images can be easily collected online, and a large example database can span new possible facial variations not sufficiently learned during the learning phase. We first extract facial features using a baseline classifier that has a certain degree of accuracy. To achieve a better performance of the proposed method, we divide the collected examples into groups using a clustering method (e.g., k-means), where each clustered group contains examples with similar characteristics. We then hierarchically partition a group formed in the previous level into other groups to analyze more specific facial characteristics, which represent an example pyramid. To describe the characteristics of a group using the clustered examples, we divide the example group into a number of sub-groups. We calculate the averages of the sub-groups and select an example most similar to the average in each sub-group because we assume that the averages of the sub-groups can directly represent their characteristics. Using the selected examples, we build example code words for a novel feature extraction. The example code words are used to measure the distances to an input image and serve as anchors to analyze a facial image in the example domain. The distance values are normalized for each group at all pyramid levels, and are concatenated to form novel features for face recognition. We verified the effectiveness of the proposed example pyramid framework using well-known proposed features, including LBP, HOG, Gabor, and the deep learning method, on the LFW database, and showed that it can yield significant improvements in recognition performance.

A novel feature selection strategy for improved face recognition in images with variations due to illumination conditions, facial expressions, and partial occlusions is presented in this dissertation. A hybrid face recognition system that uses feature maps of phase congruency and modular kernel spaces is developed. Phase congruency provides a measure that is independent of the overall magnitude of a signal, making it invariant to variations in image illumination and contrast. A novel modular kernel spaces approach is developed and implemented on the phase congruency feature maps. Smaller sub-regions from a predefined neighborhood within the phase congruency images of the training samples are merged to obtain a large set of features. These features are then projected into higher dimensional spaces using kernel methods. The unique modularization procedure developed in this research takes into consideration that the facial variations in a real world scenario are confined to local regions. The additional pixel dependencies that are considered based on their importance help in providing additional information for classification. This procedure also helps in robust localization of the variations, further improving classification accuracy. The effectiveness of the new feature selection strategy has been demonstrated by employing it in two specific applications via face authentication in low resolution cameras and face recognition using multiple sensors (visible and infrared). The face authentication system uses low quality images captured by a web camera. The optical sensor of the web camera is very sensitive to environmental illumination variations. It is observed that the feature selection policy overcomes the facial and environmental variations. A methodology based on multiple training images and clustering is also incorporated to overcome the additional challenges of computational efficiency and the subject's non involvement. A multi-sensor image fusion based face recognition methodology that uses the proposed feature selection technique is presented in this dissertation. Research studies have indicated that complementary information from different sensors helps in improving the recognition accuracy compared to individual modalities. A decision level fusion methodology is also developed which provides better performance compared to individual as well as data level fusion modalities. The new decision level fusion technique is also robust to registration discrepancies, which is a very important factor in operational scenarios. Research work is progressing to use the new face recognition technique in multi-view images by employing independent systems for separate views and integrating the results with an appropriate voting procedure.

A single gallery-based face recognition using extended joint sparse representation

Heterogeneity of a probe image is one of the most complex challenges faced by researchers and implementers of current surveillance systems. This is due to existence of multiple cameras working in different spectral ranges in a single surveillance setup. This paper proposes two different approaches including spatial sparse representations (SSR) and frequency sparse representation (FSR) to recognize on-the-move heterogeneous face images with database of single sample per person (SSPP). SCface database, with five visual and two Infrared (IR) cameras, is taken as a benchmark for experiments, which is further confirmed using CASIA NIR-VIS 2.0 face database with 17580 visual and IR images. Similarity, comparison is performed for different scenarios such as, variation of distances from a camera and variation in sizes of face images and various visual and infrared (IR) modalities. Least square minimization based approach for finding the solution is used to match face images as it makes the recognition process simpler. A side by side comparison of both the proposed approaches with the state-of-the-art, classical, principal component analysis (PCA), kernel fisher analysis (KFA) and coupled kernel embedding (CKE) methods, along with modern low-rank preserving projection via graph regularized reconstruction (LRPP-GRR) method, is also presented. Experimental results suggest that the proposed approaches achieve superior performance.

Face recognition has been a major research theme over the last two decades. There are several problems to be solved to improve the performance of face recognition. Such major problems involve appearance variation due to pose, illumination, expression, and aging. In particular, aging includes internal and external factors that cause facial appearance variation and, consequently, it is the most difficult problem to handle. In this paper, we propose a face recognition method that is robust against facial appearance variation due to aging. The proposed method employs segmentation verification of frontal face images that consists of the following three steps. (1) Face image segmentation generates three regional subimages from the input face image. (2) A matching score is calculated using gradient features from a pair consisting of the input image and a registered image for each of the three generated subimages and original (whole face) image. We obtain four matching scores. (3) The verifying classifier evaluates the matching score vector formed of the matching scores calculated for each of the four images and predicts the a posteriori probability that two matching images belong to the same person. The results of an experimental evaluation with the FGNET and MORPH face aging datasets clarify the effectiveness of the proposed method for age invariant face recognition.

Learning discriminative face features plays a major role in building high-performing face recognition models. The recent state-of-the-art face recognition solutions proposed to incorporate a fixed penalty margin on commonly used classification loss function, softmax loss, in the normalized hypersphere to increase the discriminative power of face recognition models, by minimizing the intra-class variation and maximizing the inter-class variation. Marginal penalty softmax losses, such as ArcFace and CosFace, assume that the geodesic distance between and within the different identities can be equally learned using a fixed penalty margin. However, such a learning objective is not realistic for real data with inconsistent inter-and intra-class variation, which might limit the discriminative and generalizability of the face recognition model. In this paper, we relax the fixed penalty margin constrain by proposing elastic penalty margin loss (ElasticFace) that allows flexibility in the push for class separability. The main idea is to utilize random margin values drawn from a normal distribution in each training iteration. This aims at giving the decision boundary chances to extract and retract to allow space for flexible class separability learning. We demonstrate the superiority of our ElasticFace loss over ArcFace and CosFace losses, using the same geometric transformation, on a large set of mainstream benchmarks. From a wider perspective, our ElasticFace has advanced the state-of-the-art face recognition performance on seven out of nine mainstream benchmarks. All training codes, pre-trained models, training logs will be publicly released ¹.