Supervised bilateral two-dimensional locality preserving projection algorithm based on Gabor wavelet

This work presents a model-based face recognition approach that uses a hierarchical Gabor wavelet representation and flexible neural network matching. The representation of local features is based on the Gabor wavelets transform of a number of scales and a number of orientations. The Gabor wavelet representation is used in a innovative self-organization flexible neural network matching approach that can provide robust recognition. The sparse centers of Gabor wavelets in the images and neurons placement are arranged according to the hexagonal grids. Neural network matching between the model and the input image is to find out the exact correspondence of local features and to map the model to the input image based on local similarity and neighborhood grouping of local features. Experimental results in recognizing faces that includes the variations of translation, rotation in plane, rotation in depth, and slightly changes of facial expressions are also presented.

This paper presents a novel Gabor-based kernel Principal Component Analysis (PCA) method by integrating the Gabor wavelet representation of face images and the kernel PCA method for face recognition. Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. The kernel PCA method is then extended to include fractional power polynomial models for enhanced face recognition performance. A fractional power polynomial, however, does not necessarily define a kernel function, as it might not define a positive semidefinite Gram matrix. Note that the sigmoid kernels, one of the three classes of widely used kernel functions (polynomial kernels, Gaussian kernels, and sigmoid kernels), do not actually define a positive semidefinite Gram matrix either. Nevertheless, the sigmoid kernels have been successfully used in practice, such as in building support vector machines. In order to derive real kernel PCA features, we apply only those kernel PCA eigenvectors that are associated with positive eigenvalues. The feasibility of the Gabor-based kernel PCA method with fractional power polynomial models has been successfully tested on both frontal and pose-angled face recognition, using two data sets from the FERET database and the CMU PIE database, respectively. The FERET data set contains 600 frontal face images of 200 subjects, while the PIE data set consists of 680 images across five poses (left and right profiles, left and right half profiles, and frontal view) with two different facial expressions (neutral and smiling) of 68 subjects. The effectiveness of the Gabor-based kernel PCA method with fractional power polynomial models is shown in terms of both absolute performance indices and comparative performance against the PCA method, the kernel PCA method with polynomial kernels, the kernel PCA method with fractional power polynomial models, the Gabor wavelet-based PCA method, and the Gabor wavelet-based kernel PCA method with polynomial kernels.

A bi-modal face recognition framework integrating facial expression with facial appearance

Tuning functions for automatic detection of brief changes of facial expression in the human brain

This paper introduces a novel 2D Gabor-Fisher Classifier for face recognition. The 2D-GFC method applies the 2D Fisher Linear Discriminant Analysis (2D-LDA) to the gaborfaces which is derived from the Gabor wavelets representation of face images. In our method, Gabor wavelets first derive desirable facial features characterized by spatial frequency, spatial locality, and orientation selectivity to cope with the variations due to illumination and facial expression changes. 2D-LDA is then used to enhance the face recognition performance by maximizing the Fisher's linear projection criterion. To evaluate the performance of 2D-GFC, experiments were conducted on FERET database with several other methods.

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods.

Whether the brain represents facial expressions as perceptual continua or as emotion categories remains controversial. Here, we measured the neural response to morphed images to directly address how facial expressions of emotion are represented in the brain. We found that face-selective regions in the posterior superior temporal sulcus and the amygdala responded selectively to changes in facial expression, independent of changes in identity. We then asked whether the responses in these regions reflected categorical or continuous neural representations of facial expression. Participants viewed images from continua generated by morphing between faces posing different expressions such that the expression could be the same, could involve a physical change but convey the same emotion, or could differ by the same physical amount but be perceived as two different emotions. We found that the posterior superior temporal sulcus was equally sensitive to all changes in facial expression, consistent with a continuous representation. In contrast, the amygdala was only sensitive to changes in expression that altered the perceived emotion, demonstrating a more categorical representation. These results offer a resolution to the controversy about how facial expression is processed in the brain by showing that both continuous and categorical representations underlie our ability to extract this important social cue.

In this paper, we propose a novel face recognition method based on local appearance feature extraction using dual-tree complex wavelet transform (DT-CWT). It provides a local multiscale description of images with good directional selectivity and invariance to shifts and in-plane rotations. In the dual-tree implementation, two parallel discrete wavelet transform (DWT) with different lowpass and highpass filters in different scales were used. The linear combination of subbands generated by two parallel DWT is used to generate 6 different directional subbands with complex coefficients. It is insensitive to illumination variations and facial expression changes. 2-D dual-tree complex wavelet transform is less redundant and computationally efficient. The local DT-CWT coefficients are used to extract the facial features which improve the face recognition with small sample size in less computation. The local features based methods have been successfully applied to face recognition and achieved state-of-the-art performance. Normally most of the local appearance based methods the facial features are extracted from several local regions and concatenated into an enhanced feature vector as a face descriptor. In this approach we divide the face into several (m ? m) non-overlapped parallelogram blocks instead of square or rectangle blocks. The local mean, standard deviation and energy of complex wavelet coefficients are used to describe the face image. Experiments, on two well-known databases, namely, Yale and ORL databases, shows the Local DT-CWT approach performs well on illumination, expression and perspective variant faces with single sample compared to PCA and global DT-CWT. Furthermore, in addition to the consistent and promising classification performances, our proposed Local DT-CWT based method has a really low computational complexity.

Local fusion of complex dual-tree wavelet coefficients based face recognition for single sample problem

A novel system for 3D face recognition is presented in this paper. Firstly, we reduce the noise and move spikes from all the 3D faces. Secondly, we use Iterative Closet Point (ICP) to align all 3D face with the first person, and then for each face, we find the nose tip. Once the nose tip is successfully found, we crop a region, which is defined by a sphere radius of 100 mm centered at the nose tip. Depth image are constructed using the region subsequently. Then the depth image is projected into Gabor-based Supervised Locality Sensitive Discriminant Analysis (GISLSDA) space, which is improved by Gabor wavelet and Two-Directional Two Dimensions Principal Component Analysis (2D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> PCA). Recognition is achieved by using a Nearest Neighbor (NN) classifier finally. This method is robust to changes in facial expressions and poses. The experimental results show that the new algorithm outperforms the other popular approaches reported in the literature and achieves much higher accurate recognition rate.

The panorama of face recognition is to find the dissimilarity features that can be used to discriminate individuals for their recognition. The holistic face recognition methods available in the literature perform well in the controlled environments. These methods may not appropriate to identify people through their faces in uncontrolled environments such as changes in pose, facial expression and illumination. However, the feature based methods address the issues of face recognition in uncontrolled environments to a certain extent. Therefore, the biometric researchers are trying to devise the face recognition methods that perform optimally in uncontrolled environments. This paper presents a novel method of face recognition in uncontrolled environments that works with local binary pattern of facial images and compute the dissimilarity among them using Bray Curtis dissimilarity metric. A novel concept of filtering the LBP surface texture is developed. The proposed method we called as "augmented local binary" pattern ((ALBP)) works on a combination of the principle of locality of uniform and non-uniform patterns. It replaces non-uniform patterns with the majority value of uniform patterns and combined with neighboring uniform patterns to extract valuable information regarding local descriptors. The proposed method is tested on different databases consisting uncontrolled facial images such as extended Yale B, Yale A and our database. The experimental results show that the proposed method performs better at recognizing faces in uncontrolled environments such as the facial expression, illumination, and mild pose changes.

A smile is a representative expression of happiness or high quality-of-life; however, automatic recognition of a smile according to happiness remains a challenging task. Because expressions of happiness are strongly dependent upon physical condition and occurrence of other emotions, and similar facial expression often occur under different emotions, we consider that there is no absolute visual pattern of a smile corresponding to happiness. Therefore, in this study, we assumed that a ``smile with happiness'' is observed as the temporal ascent in the degree of smiling and attempted to recognize this by capturing changes in facial expression within temporally sequential images. As an implementation of this scheme, we proposed a Siamese-structure deep neural network to compare facial expressions in two input images and estimate the existence of smile ascension or descension. For primal analysis of the proposed network, we developed a unique smiling dataset containing image pairs with various changes in smiling degree, including slight changes. The results demonstrated that the proposed method achieved nearly perfect recognition with $>0.95$ accuracy when recognizing changes in the degree of smiling that humans certainly recognize. Attention regions that contributed to the predicted labels were concentrated on the mouth, cheeks, and tail of the eyes, which indicates a reasonable function for recognizing changes in smiling degree was constructed by the proposed method.

Facial occlusions, due for example to sunglasses, hats, scarf, beards etc., can significantly affect the performance of any face recognition system. Unfortunately, the presence of facial occlusions is quite common in real-world applications especially when the individuals are not cooperative with the system such as in video surveillance scenarios. While there has been an enormous amount of research on face recognition under pose/illumination changes and image degradations, problems caused by occlusions are mostly overlooked. The focus of this paper is thus on facial occlusions, and particularly on how to improve the recognition of faces occluded by sunglasses and scarf. We propose an efficient approach which consists of first detecting the presence of scarf/sunglasses and then processing the non-occluded facial regions only. The occlusion detection problem is approached using Gabor wavelets, PCA and support vector machines (SVM), while the recognition of the non-occluded facial part is performed using block-based local binary patterns. Experiments on AR face database showed that the proposed method yields significant performance improvements compared to existing works for recognizing partially occluded and also non-occluded faces. Furthermore, the performance of the proposed approach is also assessed under illumination and extreme facial expression changes, demonstrating interesting results.

Facial expression recognition is an effective method for identifying someone's emotional expression. Emotional expressions can be recognized from changes in facial expressions, wrinkles on the forehead, blinking of the eyes, or changes in facial skin color. Facial expressions that a person generally has, such as neutral, angry, happy expressions. The problem that often occurs is the subjective assessment of a person's expression. This research examines how artificial intelligence can recognize facial expressions. The facial recognition process in the research uses a Convolutional Neural Network (CNN), which is a deep learning method capable of carrying out an independent learning process for object recognition, object extraction and classification and can be applied to high resolution images that have a nonparametric distribution model. The two main stages in CNN are feature learning and classification. The results of facial expression recognition can be used to detect a person's emotions. This research uses the FER2013 dataset which contains images of happy, sad, angry, afraid, surprised, disgusted and neutral emotions. The data set in the research received tests that had been carried out, namely the percentage of accuracy level in the model was 76%. It is hoped that the classification of emotions resulting from this research can contribute to the development of artificial intelligence technology and as a tool in various fields such as psychology, education and others. For further research, it can be developed further by adding other architectures such as VGG19, MobileNet, and ResNet-50 so that the resulting CNN model is more optimal.

This chapter assessed the possibility of using facial behaviour as another individual trait for personal identification and identification improvement. Facial expression variations were previously thought of as noise that would degrade the classification performance, so