Low Resolution Face Recognition using Enhanced SRGAN Generated Images

Modern face recognition systems extract face representations using deep neural networks (DNNs) and give excellent identification and verification results, when tested on high resolution (HR) images. However, the performance of such an algorithm degrades significantly for low resolution (LR) images. A straight forward solution could be to train a DNN, using simultaneously, high and low resolution face images. This approach yields a definite improvement at lower resolutions but suffers a performance degradation for high resolution images. To overcome this shortcoming, we propose to train a network using both HR and LR images under the guidance of a fixed network, pretrained on HR face images. The guidance is provided by minimising the KL-divergence between the output Softmax probabilities of the pretrained (i.e., Teacher) and trainable (i.e., Student) network as well as by sharing the Softmax weights between the two networks. The resulting solution is tested on down-sampled images from FaceScrub and MegaFace datasets and shows a consistent performance improvement across various resolutions. We also tested our proposed solution on standard LR benchmarks such as TinyFace and SCFace. Our algorithm consistently outperforms the state-of-the-art methods on these datasets, confirming the effectiveness and merits of the proposed method.

This paper proposes a method of synthesizing a high-resolution facial image from a low-resolution facial image based on top-down learning. A face is represented by a linear combination of prototypes of shape and texture. With the shape and texture information about the pixels in an given low-resolution facial image, we can estimate optimal coeficients for a linear combination of prototypes of shape and those of texture by solving least square minimization. Then high-resolution facial image can be synthesized by using the optimal coeficients for linear combination of the high-resolution prototypes. The encouraging results of the proposed method show that our method can be used to increase the performance of the face recognition by applying our method to enhance the low-resolution facial images captured at surveillance systems.KeywordsFace RecognitionFacial ImageInput FaceBicubic InterpolationReference FaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper proposes a new method of synthesizing a highresolution facial image from a low-resolution facial image based on top-down learning. A face is represented by a linear combination of prototypes of shape and texture. With the shape and texture information about the pixels in a given low-resolution facial image, we can estimate optimal coefficients for a linear combination of prototypes of shape and those of texture by solving least square minimization. Then high-resolution facial image can be synthesized by using the optimal coeffcients for linear combination of the high-resolution prototypes.The encouraging results of the proposed method show that our method can be used to increase the performance of the face recognition by applying our method to enhance the low-resolution facial images captured at surveillance systems.

This work proposes a new method of enhancing the resolution of facial image from a low-resolution facial image using a recursive error back-projection of example-based learning. A face is represented by a linear combination of prototypes of shape and texture. With the shape and texture information about the pixels in a given low-resolution facial image, we can estimate optimal coefficients for a linear combination of prototypes of shape and those of texture by solving least square minimization. Then high-resolution facial image can be obtained by using the optimal coefficients for linear combination of the high-resolution prototypes, in addition to, a recursive error back-projection is applied to improve the accuracy of resolution enhancement. The encouraging results of the proposed method show that our method can be used to improve the performance of the face recognition by applying our method to enhance the low-resolution facial images captured at visual surveillance systems.

An efficient face recognition system should recognize faces in different views and poses. The efficiency of a human face recognition system depends on the capability of face recognition in presence of changes in the appearance of face due to expression, pose and illumination. A novel algorithm which utilizes the combination of texture and depth information based on Modular PCA to overcome the problem of pose variation and illumination change for face recognition is proposed. The system has combined 2D and 3D systems in the feature level which presents higher performance in contrast with methods which utilizes either 2D or 3D system separately. A multimodal face recognition based on Modular PCA when compared with conventional PCA algorithm has an improved recognition rate for face images with large variations in illumination and facial expression. The proposed algorithm is tested with FRAV3D database that has faces with pose variation and illumination changes. Recognition rates from experimental results show the superiority of Modular PCA over conventional PCA methods in tackling face images with different pose variations and changes in illuminations. The proposed algorithm shows a recognition rate of 86% that is achieved in fusion experiment.

Text-guided face recognition (TGFR) aims to improve the performance of state-of-the-art face recognition (FR) algorithms by incorporating auxiliary information, such as distinct facial marks and attributes, provided as natural language descriptions. Current TGFR algorithms have been proven to be highly effective in addressing performance drops in state-of-the-art FR models, particularly in scenarios involving sensor noise, low resolution, and turbulence effects. Although existing methods explore various algorithms using different cross-modal alignment and fusion techniques, they encounter practical limitations in real-world applications. For example, during inference, textual descriptions associated with face images may be missing, lacking crucial details, or incorrect. Furthermore, the presence of inherent modality heterogeneity poses a significant challenge in achieving effective cross-modal alignment. To address these challenges, we introduce CaptionFace, a TGFR framework that integrates GPTFace, a face image captioning model designed to generate context-rich natural language descriptions from low-resolution facial images. By leveraging GPTFace, we overcome the issue of missing textual descriptions, expanding the applicability of CaptionFace to single-modal FR datasets. Additionally, we introduce a multi-scale feature alignment (MSFA) module to ensure semantic alignment between face-caption pairs at different granularities. Furthermore, we introduce an attribute-aware loss and perform knowledge adaptation to specifically adapt textual knowledge from facial features. Extensive experiments on three face-caption datasets and various unconstrained single-modal benchmark datasets demonstrate that CaptionFace significantly outperforms state-of-the-art FR models and existing TGFR approaches.

In this paper, we study face hallucination, or synthesizing a high-resolution face image from an input low-resolution image, with the help of a large collection of other high-resolution face images. Our theoretical contribution is a two-step statistical modeling approach that integrates both a global parametric model and a local nonparametric model. At the first step, we derive a global linear model to learn the relationship between the high-resolution face images and their smoothed and down-sampled lower resolution ones. At the second step, we model the residue between an original high-resolution image and the reconstructed high-resolution image after applying the learned linear model by a patch-based non-parametric Markov network to capture the high-frequency content. By integrating both global and local models, we can generate photorealistic face images. A practical contribution is a robust warping algorithm to align the low-resolution face images to obtain good hallucination results. The effectiveness of our approach is demonstrated by extensive experiments generating high-quality hallucinated face images from low-resolution input with no manual alignment.

This paper proposes a new method of enhancing the resolution of low-resolution facial images using an error back-projection method based on a top-down learning. A face is represented by a linear combination of prototypes of shape and texture. With the shape and texture information about the pixels in a given low-resolution facial image, we can estimate optimal coefficients for a linear combination of prototypes of shape and those of texture by solving least square minimization. Then high-resolution facial image can be obtained by using the optimal coefficients for linear combination of the high-resolution prototypes. In addition, an error back-projection procedure is applied to improve the accuracy of resolution enhancement.

This paper proposes a new method of enhancing the resolution of low-resolution facial images using an error back-projection method based on a top-down learning. A face is represented by a linear combination of prototypes of shape and texture. With the shape and texture information about the pixels in a given low-resolution facial image, we can estimate optimal coefficients for a linear combination of prototypes of shape and those of texture by solving least square minimization. Then high-resolution facial image can be obtained by using the optimal coefficients for linear combination of the high-resolution prototypes. In addition, an error back-projection procedure is applied to improve the accuracy of resolution enhancement.

Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one gallery per subject only. With the easy access to high-resolution (HR) face images nowadays, some HR face databases have recently been developed. However, few studies have tackled the use of HR information for face recognition or verification. In this paper, we propose a pose-invariant face-verification method, which is robust to alignment errors, using the HR information based on pore-scale facial features. A new keypoint descriptor, namely, pore-Principal Component Analysis (PCA)-Scale Invariant Feature Transform (PPCASIFT)-adapted from PCA-SIFT-is devised for the extraction of a compact set of distinctive pore-scale facial features. Having matched the pore-scale features of two-face regions, an effective robust-fitting scheme is proposed for the face-verification task. Experiments show that, with one frontal-view gallery only per subject, our proposed method outperforms a number of standard verification methods, and can achieve excellent accuracy even the faces are under large variations in expression and pose.

Low resolution severely degrades the detail of face images, while bad illumination condition further increases the difficulty of face recognition. In this paper, we present a new differential images based face recognition approach to deal with low resolution and variable illumination problems. Firstly, we resort to down-sampling and interpolation to build differential image database. Then feature extraction is preformed to project the differential images into differential feature subspace. For recognizing input low-resolution (LR) face images that suffer from illumination variation, normalization are employed to compensate the illumination of LR images and hallucination method is to infer the corresponding project coefficients in the differential feature subspace. Finally, similarity measure is used for face discrimination. Different from Eigenspace-based face recognition techniques, the utilization of differential images instead of original images ensures the distinction between face images of different subjects more effectively. Experiments demonstrate that the proposed approach can improve recognition rate of LR face under variable illumination significantly.

With the recent increase in intelligent CCTVs for visual surveillance, a new image degradation that integrates resolution conversion and synthetic rain models is required. For example, in heavy rain, face images captured by CCTV from a distance have significant deterioration in both visibility and resolution. Unlike traditional image degradation models (IDM), such as rain removal and super resolution, this study addresses a new IDM referred to as a scale-aware heavy rain model and proposes a method for restoring high-resolution face images (HR-FIs) from low-resolution heavy rain face images (LRHR-FI). To this end, a two-stage network is presented. The first stage generates low-resolution face images (LR-FIs), from which heavy rain has been removed from the LRHR-FIs to improve visibility. To realize this, an interpretable IDM-based network is constructed to predict physical parameters, such as rain streaks, transmission maps, and atmospheric light. In addition, the image reconstruction loss is evaluated to enhance the estimates of the physical parameters. For the second stage, which aims to reconstruct the HR-FIs from the LR-FIs outputted in the first stage, facial component-guided adversarial learning (FCGAL) is applied to boost facial structure expressions. To focus on informative facial features and reinforce the authenticity of facial components, such as the eyes and nose, a face parsing-guided generator and facial local discriminators are designed for FCGAL. The experimental results verify that the proposed approach based on a physical-based network design and FCGAL can remove heavy rain and increase the resolution and visibility simultaneously. Moreover, the proposed heavy rain face image restoration outperforms state-of-the-art models of heavy rain removal, image-to-image translation, and super resolution.

Face Super-Resolution (FSR) aims to infer High-Resolution (HR) face images from the captured Low-Resolution (LR) face image with the assistance of external information. Existing FSR methods are less effective for the LR face images captured with serious low-quality since the huge imaging/degradation gap caused by the different imaging scenarios (i.e., the complex practical imaging scenario that generates test LR images, the simple manual imaging degradation that generates the training LR images) is not considered in these algorithms. In this paper, we propose an image homogenization strategy via re-expression to solve this problem. In contrast to existing methods, we propose a homogenization projection in LR space and HR space as compensation for the classical LR/HR projection to formulate the FSR in a multi-stage framework. We then develop a re-expression process to bridge the gap between the complex degradation and the simple degradation, which can remove the heterogeneous factors such as serious noise and blur. To further improve the accuracy of the homogenization, we extract the image patch set that is invariant to degradation changes as Robust Neighbor Resources (RNR), with which these two homogenization projections re-express the input LR images and the initial inferred HR images successively. Both quantitative and qualitative results on the public datasets demonstrate the effectiveness of the proposed algorithm against the state-of-the-art methods.

Detecting human faces in low-resolution images is more difficult than high quality images because people appear smaller and facial features are not as clear as high resolution face images. Furthermore, the regions of interest are often impoverished or blurred due to the large distance between the camera and the objects which can decrease detection rate and increase false alarms. As a result, the performance of face detection (detection rate and the number of false positives) in low-resolution images can affect directly subsequent applications such as face recognition or face tracking. In this paper, a novel method, based on cascade Adaboost and Histogram of Oriented Gradients (HOG), is proposed to improve face detection performance in low resolution images, while most of researches have been done and tested on high quality images. The focus of this work is to improve the performance of face detection by increasing the detection rate and at the same time decreasing the number of false alarms. The concept behind the proposed combination is based on the a-priori rejection of false positives for a more accurate detection. In other words in order to increase human face detection performance, the first stage (cascade Adaboost) removes the majority of the false alarms while keeping the detection rate high, however many false alarms still exist in the final output. To remove existing false alarms, a stage (HOG+SVM) is added to the first stage to act as a verification module for more accurate detection. The method has been extensively tested on the Carnegie Melon University (CMU) database and the low-resolution images database. The results show better performance compared with existing techniques.

This work presents a method to automatically locate facial feature points under large variations in pose, illumination and facial expressions. First we propose a method to calculate probabilistic-like output for each pixel of image. This probabilistic-like output describes the possibility of the pixel to be the center of specified object. A Gaussian mixture model is used to approximate the distribution of probabilistic-like output. The centers of these Gaussians are assigned with a probabilistic-like measure and they are considered as candidate feature points. There might be one or more candidate feature points in each facial region. A 3D model of facial feature points is built to enforce constraints on the localization results of feature points. Compared with active shape model (ASM) and its variant methods, our method could accommodate larger variations in pose, lighting and face expressions. Moreover, it is less sensitive to initialization errors, accurate, and fast. It takes a computer with P4 CPU about 10 ms to locate the five feature points (two eye centers, two mouth corners and nose tip). The feature localization accuracy is comparable with the accuracy of manually labeled features and it is robust to noise (glasses, beards). Experiments on FERET gallery and PIE are reported in this paper as well.