Facial Expression Recognition Using Kernel Discriminant Plane

Texture characteristics among the landmark points reflected in human faces are important features in the recognition of facial expression. In this paper, we propose a salient feature based facial expression on texture characteristics. Detection of effective landmarks are performed by appearance based models and corresponding texture regions are extracted from face images. Local Binary Pattern (LBP) is used to compute the texture feature. Texture feature is used for normalizing texture signatures. In addition, we introduce stability indices corresponding to texture features which are exploited as features in the present scope. Moreover, statistical features such as moment, skewness, kurtosis and entropy are also supplemented to the feature set. These salient features together are used as an input to Nonlinear AutoRegressive with eXogenous (NARX) for recognition of the human facial expression. The Cohn-Kanade (CK+), Japanese Female Facial Expression (JAFFE), MMI and MUG benchmark databases are used to conduct the experiments and the results justify the effectiveness of the proposed procedure.

Facial expressions are important in people’s daily communications. Recognising facial expressions also has many important applications in the areas such as healthcare and e-learning. Existing facial expression recognition systems have problems such as background interference. Furthermore, systems using traditional approaches like SVM (Support Vector Machine) have weakness in dealing with unseen images. Systems using deep neural network have problems such as requirement for GPU, longer training time and requirement for large memory. To overcome the shortcomings of pure deep neural network and traditional facial recognition approaches, this paper presents a new facial expression recognition approach which has image pre-processing techniques to remove unnecessary background information and combines deep neural network ResNet50 and a traditional classifier-- the multiclass model for Support Vector Machine to recognise facial expressions. The proposed approach has better recognition accuracy than traditional approaches like Support Vector Machine and doesn’t need GPU. We have compared 3 proposed frameworks with a traditional SVM approach against the Karolinska Directed Emotional Faces (KDEF) Database, the Japanese Female Facial Expression (JAFFE) Database and the extended Cohn-Kanade dataset (CK+), respectively. The experiment results show that the features extracted from the layer 49Relu have the best performance for these three datasets.

Automated recognition of facial expression has attracted significant attention in recent years due to its potential applicability in security and surveillance, human computer interaction, social robotics, and animation. This paper presents a new facial expression recognition method that utilizes bit plane specific local image description in a weighted score level fusion. The motivation is to utilize bit plane slicing to highlight the contribution of a particular bit plane made to the holistic facial appearance, which is then used in a weighted score level fusion in order to boost the recognition performance. A new local image descriptor is proposed specifically to extract local features from bit plane representations that utilizes Fisher linear discriminant to maximize the inter-class distance, while minimizing the intra-class variance. Two well-known facial expression databases, namely the Cohn-Kanade (CK) and the Japanese female facial expression (JAFFE) database have been used to evaluate the performance of the proposed method against existing facial appearance descriptors, such as local binary pattern (LBP), local ternary pattern (LTP), local directional pattern (LDP), and linear discriminant analysis (LDA). Experiments with a total of seven prototypic facial expressions show promising results for the proposed method, as compared with the other existing methods.

In this paper, we proposed a novel algorithm for Facial Expression Recognition (FER) which was based on fusion of gabor texture features and Local Phase Quantization (LPQ). Firstly, the LPQ feature and gabor texture feature were respectively extracted from every expression image. LPQ features are histograms of LPQ transform. Five scales and eight orientations of gabor wavelet filters are used to extract gabor texture features and adaboost algorithm is used to select gabor features. Then we obtain two expression recognition results on both expression features by Sparse Representation-based Classification (SRC) method. Finally, the final expression recognition was performed by fusion of residuals of two SRC algorithms. The experiment results on Japanese Female Facial Expression (JAFFE) database demonstrated that the new algorithm was better than the original two algorithms, and this algorithm had a much higher recognition rate than the traditional algorithm.

Facial expression recognition is one of the most important tasks in human–computer interaction, affective computing, computer vision, and related work. Feature additive pooling and progressive fine‐tuning of the convolutional neural network (CNN) for facial expression recognition in a static image are introduced. Network is proposed that partially employs the visual geometry group (VGG)‐face model pre‐trained on a VGG‐face dataset. The characteristics and distribution of the facial expression images in each database are biased according to the purpose of the publicly available facial database used. To alleviate this problem, a CNN model is developed that merges progressively fine‐tuned CNNs into a single network. Experiments were carried out to validate the presented method using facial expression images from the Cohn–Kanade +, Karolinska directed emotional face, and Japanese female facial expression databases, and cross‐database evaluation results show that the method is superior to state‐of‐the‐art methods.

The main goal of this work is to develop a fully automatic face recognition algorithm. Scale Invariant Feature Transform (SIFT) has sparingly been used in face recognition. In this paper, a Modified SIFT (MSIFT) approach has been proposed to enhance the recognition performance of SIFT. In this paper, the work is done in three steps. First, the smoothing of the image has been done using DWT. Second, the computational complexity of SIFT in descriptor calculation is reduced by subtracting average from each descriptor instead of normalization. Third, the algorithm is made automatic by using Coefficient of Correlation (CoC) instead of using the distance ratio (which requires user interaction). The main achievement of this method is reduced database size, as it requires only neutral images to store instead of all the expressions of the same face image. The experiments are performed on the Japanese Female Facial Expression (JAFFE) database, which indicates that the proposed approach achieves better performance than SIFT based methods. In addition, it shows robustness against various facial expressions.

Certain situations require human intervention to identify the emotional condition of the subject. In order to automate this process, the paper explores the idea of applying the Fisher face method to build an accurate, reliable and automatic system accompanied by supervised classification methods such as K-nearest neighbor and artificial neural network for human emotion recognition as depicted in still images. The experimental analysis is performed on the popular Japanese female facial expression (JAFFE) database. We have explored the efficacy of the Fisher face model in classifying human emotion based on content obtained from facial images. We have used basic prototypic emotions as defined by Paul Ekman - happiness (H), sadness (S), surprise (SU), and neutral (N), as our four classes. The technique used for feature extraction of facial expression detection is local fisher discriminant analysis (LFDA) which performs dimensionality reduction. The supervised classification task is carried out by K-nearest neighbor (KNN) and artificial neural network (ANN) separately. It is observed that ANN performance is better in case of emotions like happiness and surprise while neutral and sad emotions are recognized better by KNN technique.

A novel approach to facial expression recognition (FER) based on the combination of Non-negative Matrix Factorization (NMF) and Support Vector Machine (SVM) was proposed. One key step in FER is to extract expression features from the original face images. NMF is an effective approach to extract expression features because NMF decomposition makes the reconstruction of expression images in a non-subtractive way and is much similar to the process of forming unity from parts. The proposed algorithm first processes facial expression image with histogram equalization operator. Then NMF method is used for feature dimension reduction and SVM for classification. Finally, the algorithm was implemented with Matlab and experimented in Japanese female facial expression database (JAFEE database). A recognition rate of 66.19% was obtained and showed the effectiveness of the proposed algorithm

Facial Expression Recognition (FER) is a critical component of affective computing, with deep learning models dominating performance metrics. In contrast, geometric approaches based on the Facial Action Coding System (FACS) offer explainability through using triangles aligned to facial landmarks. The notable points of these triangles capture the deformation of muscles. However, restricting the feature extraction to notable points may be suboptimal. This paper introduces a novel method for optimizing the extraction of features by searching for optimal inner points in 22 facial triangles applying three metaheuristics: Differential Evolution (DE), Particle Swarm Optimization (PSO), and Convex Partition (CP). This results in a set of 59 geometric-based descriptors that capture muscle deformation more accurately. The proposed method was evaluated using five machine learning classifiers on two benchmark databases: the Karolinska Directed Emotional Faces (KDEF) and the Japanese Female Facial Expression (JAFFE). Experimental results demonstrate significant performance improvements. The combination of DE with a Multi-Layer Perceptron (MLP) achieved an accuracy of 0.91 on the KDEF database, while Support Vector Machine (SVM) optimized via CP attained an accuracy of 0.81 on the JAFFE database. Statistical analysis confirms that optimized descriptors yield higher accuracy than previous geometric methods.

There have been quite a few studies on facial expression recognition over the years, and it is still a challenging subject due to the significant inter-class variability. Facial expression research in this field focuses on the development of techniques to identify, code, and extract facial expressions to improve prediction by computer. With great success of machine learning, the various texture descriptors are exploited to obtain a better performance. This paper proposes a method based on the aggregation between different descriptors Histogram of oriented Gradient (HOG) and Local Binary Pattern (LBP). First stage the input image has pre-processed to detect dace area which helps to extract most significant features. Then, Diagonal-HOG (D-HOG) also has extracted and aggregated all features. Finally, Support Vector Machine (SVM) has been used a classifier to classify each feature as well as aggregated features. We evaluate our method using Japanese Female Facial Expressions database (JAFFE), experimental results showed that the proposed method is accurate and efficient in recognizing facial expressions.

Recognition of human faces is a very important task in many applications such as authentication and surveillance. An efficient face recognition system with face image representation using averaged wavelet and wavelet packet coefficients, Discriminative Common Vector (DCV) and modified Local Binary Patterns (LBP) and recognition using radial basis function (RBF) network is presented. Face images are decomposed by 2-level wavelet and wavelet packet transformation. The discriminative common vectors are obtained for averaged wavelet. The new proposed LBP operator is applied on the obtained DCV and also applied on averaged wavelet packet coefficients of all the samples of a class. The histogram values obtained from the LBP are recognized using RBF network. The proposed work is tested on three face databases such as Olivetti Oracle Research Lab (ORL), Japanese Female Facial Expression (JAFFE) and Essex face database. The proposed method results in good recognition rates along with less training time because of the extracted discriminant input from the preprocessing steps involved in the proposed work.

In this paper, a new method for extracting expression-independent face features based on HOSVD (Higher-Order Singular Value Decomposition) is proposed and used for face recognition. In the new method, it is assumed that a facial expression could be represented by the facial expressions in the training set. In addition, the expression with higher similarity to the expression of test person has higher probability to represent the expression of test person. Expression-similarity weighted expression feature, which is the optimal estimation based on Bayesian estimation theory and the assumption, is used to estimate the face feature of the test person. As a result, the estimated face feature can reduce the influence of expression caused by insufficient training data and becomes less expression-dependent, and can be more robust to new expressions. The proposed method has been applied to Japanese Female Facial Expression (JAFFE) database. Expression-independent experimental results show the superiority of proposed method over the existing methods in terms of recognition rate and accumulative recognition rate.

Texture is the surface property that is used to identify and recognise objects. This property is widely used in many applications including texture-based face recognition systems, surveillance, identity verification and so on. The Local binary pattern (LBP) texture method is most successful for face recognition. Owing to the great success of LBP, recently many models, which are variants of LBP have been proposed for texture analysis. Some of the derivatives of LBPs are multivariate local binary pattern, centre symmetric local binary pattern, local binary pattern variance, dominant local binary pattern, advanced local binary pattern, local texture pattern (LTP) and local derivative pattern (LDP). In this scenario, it is essential to review, whether LBP or their derivatives perform better for face recognition. The real-time challenges such as illumination changes, rotations, angle variations and facial expression variations are evaluated by different LBP-based models. Experiments were conducted on the Japanese female facial expression, YALE and FRGC version2 databases. The results show that LDP and LTP perform much better than the other LBP-based models.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Recognizing facial expression has attracted much more attention due to its broad range of applications in human–computer interaction systems. Although facial representation is crucial to final recognition accuracy, traditional handcrafted representations only reflect shallow characteristics and it is uncertain whether the convolutional layer can extract better ones. In addition, the policy that weights are shared across a whole image is improper for structured face images. To overcome such limitations, a novel method based on patches of interest, the Patch Attention Layer (PAL) of embedding handcrafted features, is proposed to learn the local shallow facial features of each patch on face images. Firstly, a handcrafted feature, Gabor surface feature (GSF), is extracted by convolving the input face image with a set of predefined Gabor filters. Secondly, the generated feature is segmented as nonoverlapped patches that can capture local shallow features by the strategy of using different local patches with different filters. Then, the weighted shallow features are fed into the remaining convolutional layers to capture high-level features. Our method can be carried out directly on a static image without facial landmark information, and the preprocessing step is very simple. Experiments on four databases show that our method achieved very competitive performance (Extended Cohn–Kanade database (CK+): 98.93%; Oulu-CASIA: 97.57%; Japanese Female Facial Expressions database (JAFFE): 93.38%; and RAF-DB: 86.8%) compared to other state-of-the-art methods.