Facial Expression Recognition Based on Feature Fusion and Sparse Representation

In this paper, we present a facial expression recognition method based on Gabor feature and sparse representation. Sparse Representation based Classification (SRC) has been widely used in computer vision and pattern recognition. And Gabor filter banks can be used to approximately model the signal processing in visual primary cortex. We believe that the nature of the attractive performance of SRC and Gabor feature lies in that they both followed the natures of signal perception of retina and information processing of cortex in human vision. Therefore, we combined the Gabor feature and SRC for facial expression recognition. The comparison experiments of proposed Gabor+SRC algorithm and straightforward SRC application are conducted on JAFFE database. And the experimental results showed the attractive performance of the proposed Gabor+SRC method.

In this paper, we proposed a novel algorithm for Facial Expression Recognition (FER) which was based on fusion of gabor texture features and Local Phase Quantization (LPQ). Firstly, the LPQ feature and gabor texture feature were respectively extracted from every expression image. LPQ features are histograms of LPQ transform. Five scales and eight orientations of gabor wavelet filters are used to extract gabor texture features and adaboost algorithm is used to select gabor features. Then we obtain two expression recognition results on both expression features by Sparse Representation-based Classification (SRC) method. Finally, the final expression recognition was performed by fusion of residuals of two SRC algorithms. The experiment results on Japanese Female Facial Expression (JAFFE) database demonstrated that the new algorithm was better than the original two algorithms, and this algorithm had a much higher recognition rate than the traditional algorithm.

Robust automatic facial expression detection method based on sparse representation plus LBP map

The theory of compressive sensing applies the sparse representation to the extraction of useful information from signals and brings a breakthrough to the theory of signal sampling. Based on compressive sensing, sparse representation-based classification (SRC) is proposed. SRC uses the compressibility of the image data to represent the facial image sparsely and could solve the problems of both massive calculation and information loss in dealing with signals. SRC does not, however, deal with the effects of variable illumination, posture and incomplete face image, which could result in severe performance degradation. This paper studies the differences between SRC recognition and human recognition. We find that there is an obvious disadvantage in the SRC algorithm, and it will significantly affect the face recognition performance in actual environment, especially for the variable illumination, posture and incomplete face image. To overcome the disadvantage of SRC algorithm, we propose an SRC-based twice face recognition algorithm named T_SRC. T_SRC uses bidirectional PCA, linear discriminant analysis and GradientFace to execute multichannel analysis, which could extract more "holistic/configural" face features in actual environment than by using SRC algorithm directly. Based on the multichannel analysis, we identify the test image by SRC firstly. Then, by analyzing the residual, this algorithm could decide whether the twice recognition is needed. If the twice recognition is needed, T_SRC extracts the facial details ("featural" face features) by the improved Harris point and Gabor filter detector. We suppose that the facial details are more stable than the whole face in actual environment, and later experiments verify our assumption. At last, this algorithm identifies the class of the test image by SRC again. The results of the experiments prove that the T_SRC algorithm has better recognition rate than SRC.

Sparse representation based classification (SRC) and collaborative representation based classification (CRC) are two well-known methods in representation-based classification for face recognition. SRC emphasizes on the role of sparsity in coding, while CRC obtains an enhanced sparse representation with collaborative contributions from all classes. Previous studies show that sparsity of representation coefficients of SRC and CRC is significant for robust classification that may amplify the nonlocal contribution from other classes. However, the parameters are closely associated with the sparsity and it is very difficult to solve the optimal solutions. This paper proposed a novel representation-based image classification method that integrates SRC with CRC by a multiplication operation on the solutions to obtain enhanced sparse representation. We conducted experiments on several benchmark face databases and the experimental results showed that our algorithm produces higher accuracy on classification than both SRC and CRC.

Sparse representation based classification (SRC) has gained great success in image recognition. In this paper, we propose a generalized CRC method, essentially a SRC using Euler sparse representation, for image classification. To be specific, our CRC first maps the image into a complex space by Euler representation with a negligible effect for outliers and illumination, and then performs a complex SRC method. We also present a fast and efficient algorithm to solve Euler SRC subproblems. Extensive experimental results illustrate that our method outperforms traditional SRC methods and achieves better performance for image classification.

The sparse representation based classification (SRC) can effectively improve the face recognition rate. Smoothed l 0 algorithm has much faster calculation speed and requires fewer measured values than the other sparse representation method. In this paper, the sparse representation and smoothed l 0 algorithm for face recognition are presented to improve the face recognition under various conditions such as face disguise, illumination and pose changes, etc. The experiments on the AR, Extended Yale B and FERET face database verify the effectiveness of the presented method. The experimental results show that the face recognition algorithm increases to a certain extent in terms of recognition robustness and time than the original SRC.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

We develop a new method to recognize facial expressions. Sparse representation based classification (SRC) is used as the classifier in this method, because of its robustness to occlusion. Histograms of Oriented Gradient (HOG) descriptors and Local Binary Patterns are used to extract features. Since the results of HOG+SRC and LBP+SRC are complimentary, we use a classifier combination strategy to fuse these two results. Experiments on Cohn-Kanade database show that the proposed method gives better performance than existing methods such as Eigen+SRC, LBP+SRC and so on. Furthermore, the proposed method is robust to assigned occlusion.Keywords Terms-facial expression recognitionHOG descriptorsLocal binary patternClassifiers combinationSparse representation based classification (SRC)assigned occlusion

Recently, facial expression recognition has attracted much attention in machine vision research because of its various applications. Accordingly, many facial expression recognition systems have been proposed. However, the majority of existing systems suffer from a critical problem: geometric variability. It directly affects the performance of geometric feature-based facial expression recognition approaches. Furthermore, it is a crucial challenge in appearance feature-based techniques. This variability appears in both neutral faces and facial expressions. Appropriate face geometry normalization can improve the accuracy of each facial expression recognition system. Therefore, this paper proposes different geometric models or shapes for normalization. Face geometry normalization removes geometric variability of facial images and consequently, appearance feature extraction methods can be accurately utilized to represent facial images. Thus, some expression-based geometric models are proposed for facial image normalization. Next, local binary patterns and local phase quantization are used for appearance feature extraction. A combination of an effective geometric normalization with accurate appearance representations results in more than a 4% accuracy improvement compared to several state-of-the-arts in facial expression recognition. Moreover, utilizing the model of facial expressions which have larger mouth and eye region sizes gives higher accuracy due to the importance of these regions in facial expression.

Recently the sparse representation based classification (SRC) has been successfully used in face recognition(FR). Using this Algorithm firstly we should code a query sample as a sparse linear combination of all the training samples. Secondly we need to classify it by evaluating which class leads to the minimum representation error. The dimension of imagery data is typically very high and that makes it computationally costly to process high-resolution images. It is widely believed that the - norm sparsity[4] constraint on coding coefficients plays a key role in the success of SRC. The theory of compressed sensing(CS) offers an useful method to reduce the face dimension. The basic principles of CS can reduce much lower-dimensional measurements of the images, without significantly compromising recognition performance. In order to get the Optimal sparse solution, we need to improve the sparse representation Algorithm.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

This paper presents a method to perform sparse representation based classification (SRC) in a more accurate and efficient way. In this method, training data is first mapped into different feature spaces and multiple dictionaries are built by utilizing a Fisher discriminative based method. These dictionaries can be considered as efficient representations of the data which are then used in a multimodal SRC framework to classify test samples. In comparison to the original SRC method where only one modality of training space is utilized, the proposed method classifies test samples in a more accurate and efficient way. Experimental results from two different face datasets show that the proposed multimodal method has higher recognition rate compared to single-modality SRC based methods. The accuracy of the proposed method is also compared to other multi-modality classifiers and the results confirm that higher recognition rates are achieved in comparison with other common classification algorithms.

A novel SRC fusion method using hierarchical multi-scale LBP and greedy search strategy

A new method based on Monogenic Binary Coding (MBC) is proposed for facial expression feature extraction and representation. Firstly, monogenic signal analysis is used to extract multi-scale magnitude, orientation and phase components. Secondly, Monogenic Binary Coding (MBC) is used to encode the monogenic local variation and intensity in local regions of each extracted component in each scale and local histograms are built. Then Blocked Fisher Linear Discrimination (BFLD) is used to reduce the dimensionality of histogram features and to enhance discrimination. Finally the three complementary components are fused for more effective facial expression recognition (FER). Experiment results on Japanese female expression database (JAFFE) show that the performance of the fusion method is even better than state-of-the-art local feature based FER methods such as Local Binary Pattern (LBP)+Sparse Representation (SRC), Local Phase Quantization (LPQ)+SRC ,etc.