Facial Expression Recognition: a Comparison with Different Classical and Deep Learning Methods

INTRODUCTION: As one of the important research directions in the field of computer vision, facial emotion recognition plays an important role in people's daily life. How to make the computer accurately read facial emotion is an important research content.OBJECTIVES: In the current research on facial emotion recognition, there are some problems such as poor generalization ability of network model and low robustness of recognition system. To solve above problems, we propose a novel facial emotion recognition method.METHODS: Our method of feature extraction using the stationary wavelet entropy, which combines single hidden layer feedforward neural network with biogeography-based optimization for facial emotion recognition.RESULTS: The simulation results show that the overall accuracy of our method is 93.79±1.24%.CONCLUSION: This model is superior to the current mainstream facial emotion recognition models in the performance of facial emotion detection. In future research, we will try deep learning and other training methods.

A novel feature extraction method that utilizes nonlinear mapping from the original data space to the feature space is presented in this dissertation. Feature extraction methods aim to find compact representations of data that are easy to classify. Measurements with similar values are grouped to same category, while those with differing values are deemed to be of separate categories. For most practical systems, the meaningful features of a pattern class lie in a low dimensional nonlinear constraint region (manifold) within the high dimensional data space. A learning algorithm to model this nonlinear region and to project patterns to this feature space is developed. Least squares estimation approach that utilizes interdependency between points in training patterns is used to form the nonlinear region. The proposed feature extraction strategy is employed to improve face recognition accuracy under varying illumination conditions and facial expressions. Though the face features show variations under these conditions, the features of one individual tend to cluster together and can be considered as a neighborhood. Low dimensional representations of face patterns in the feature space may lie in a nonlinear constraint region, which when modeled leads to efficient pattern classification. A feature space encompassing multiple pattern classes can be trained by modeling a separate constraint region for each pattern class and obtaining a mean constraint region by averaging all the individual regions. Unlike most other nonlinear techniques, the proposed method provides an easy intuitive way to place new points onto a nonlinear region in the feature space. The proposed feature extraction and classification method results in improved accuracy when compared to the classical linear representations. Face recognition accuracy is further improved by introducing the concepts of modularity, discriminant analysis and phase congruency into the proposed method. In the modular approach, feature components are extracted from different sub-modules of the images and concatenated to make a single vector to represent a face region. By doing this we are able to extract features that are more representative of the local features of the face. When projected onto an arbitrary line, samples from well formed clusters could produce a confused mixture of samples from all the classes leading to poor recognition. Discriminant analysis aims to find an optimal line orientation for which the data classes are well separated. Experiments performed on various databases to evaluate the performance of the proposed face recognition technique have shown improvement in recognition accuracy, especially under varying illumination conditions and facial expressions. This shows that the integration of multiple subspaces, each representing a part of a higher order nonlinear function, could represent a pattern with variability. Research work is progressing to investigate the effectiveness of subspace projection methodology for building manifolds with other nonlinear functions and to identify the optimum nonlinear function from an object classification perspective.

Face recognition is efficiently used in computer vision, pattern recognition, biometric, law enforcement, surveillance, criminal investigations, and missing person detection. However, face recognition is still a challenging task under variant illuminations, pose, expression, occlusion, and background. In this paper, we proposed a cascading of histogram of oriented gradients (HOG) and local binary pattern (LBP) feature extraction method on ORL face dataset, which can improve recognition rates while also addressing pose, scale, expression, and variant illumination issues. The facial recognition network was trained using various classifiers such as with KNN, SVM, and random forest (RF). The results have been shown that the accuracy rate of the combined LBP and HOG feature extraction method is better than individual LBP or HOG features for ORL database face recognition.

Facial expression recognition via ResNet-50

One of the most common causes of death worldwide is skin cancer and its incidence is increasing. To achieve optimal treatment and improve clinical outcomes for patients, precision skin cancer detection and classification approaches are required, which can be achieved through the application of feature extraction and machine learning algorithms. The development of such algorithms to identify important patterns from skin cancer image datasets enables early detection and more accurate classification and more effective treatment. Although previous studies have tried to detect skin cancer using feature extraction techniques such as HFF, HOG, and GLCM, some weaknesses still need to be improved. This research aims to combine various feature extraction methods such as Gray Level Co-occurrence Matrix, Histogram Oriented Gradients, and Local Binary Patterns and machine learning algorithms such as Support Vector Machine, Random Forest, and Gaussian Naïve Bayes in the classification process between Melanoma and Nevus skin cancers. In this research, the number of datasets used is 17,397 derived from the ISIC Dataset. The results showed that the Histogram Oriented Gradients method with Support Vector Machine algorithm achieved the highest accuracy of 92%. The combination of Gray Level Co-occurrence Matrix and Local Binary Patterns with Random Forest algorithm also achieved an accuracy of 92%, the combination of Gray Level Co-occurrence Matrix, Histogram Oriented Gradients, and Local Binary Patterns with Random Forest algorithm also resulted in an accuracy of 92%. These findings suggest that the combination of various feature extraction methods and machine learning algorithms can improve accuracy in skin cancer classification, which in turn can contribute to early detection and more effective treatment.

The potential of facial and facial expression recognitions has gained increased interest in social interactions and biometric identification. Earlier facial identification methods suffer from drawbacks due to the lower identification accuracy under difficult lighting conditions. This paper presents two novel new descriptors called Histogram of Directional Gradient (HDG) and Histogram of Directional Gradient Generalized (HDGG) to extracting discriminant facial expression features for better classification accuracy with good efficiency than existing classifiers. The proposed descriptors are based on the directional local gradients combined with SVM (Support Vector Machine) linear classification. To build an efficient face and facial expression recognition, features with reduced dimension are used to boost the performance of the classification. Experiments are conducted on two public-domain datasets: JAFFE for facial expression recognition and YALE for face recognition. The experiment results show the best overall accuracy of 92.12% compared to other existing works. It demonstrates a fast execution time for face recognition ranging from 0.4 to 0.7 s in all evaluated databases.

The surface electromyography (sEMG) technique is proposed for muscle activation detection and intuitive control of prostheses or robot arms. Motion recognition is widely used to map sEMG signals to the target motions. One of the main factors preventing the implementation of this kind of method for real-time applications is the unsatisfactory motion recognition rate and time consumption. The purpose of this paper is to compare eight combinations of four feature extraction methods (Root Mean Square (RMS), Detrended Fluctuation Analysis (DFA), Weight Peaks (WP), and Muscular Model (MM)) and two classifiers (Neural Networks (NN) and Support Vector Machine (SVM)), for the task of mapping sEMG signals to eight upper-limb motions, to find out the relation between these methods and propose a proper combination to solve this issue. Seven subjects participated in the experiment and six muscles of the upper-limb were selected to record sEMG signals. The experimental results showed that NN classifier obtained the highest recognition accuracy rate (88.7%) during the training process while SVM performed better in real-time experiments (85.9%). For time consumption, SVM took less time than NN during the training process but needed more time for real-time computation. Among the four feature extraction methods, WP had the highest recognition rate for the training process (97.7%) while MM performed the best during real-time tests (94.3%). The combination of MM and NN is recommended for strict real-time applications while a combination of MM and SVM will be more suitable when time consumption is not a key requirement.

Recently, facial expression recognition has attracted much attention in machine vision research because of its various applications. Accordingly, many facial expression recognition systems have been proposed. However, the majority of existing systems suffer from a critical problem: geometric variability. It directly affects the performance of geometric feature-based facial expression recognition approaches. Furthermore, it is a crucial challenge in appearance feature-based techniques. This variability appears in both neutral faces and facial expressions. Appropriate face geometry normalization can improve the accuracy of each facial expression recognition system. Therefore, this paper proposes different geometric models or shapes for normalization. Face geometry normalization removes geometric variability of facial images and consequently, appearance feature extraction methods can be accurately utilized to represent facial images. Thus, some expression-based geometric models are proposed for facial image normalization. Next, local binary patterns and local phase quantization are used for appearance feature extraction. A combination of an effective geometric normalization with accurate appearance representations results in more than a 4% accuracy improvement compared to several state-of-the-arts in facial expression recognition. Moreover, utilizing the model of facial expressions which have larger mouth and eye region sizes gives higher accuracy due to the importance of these regions in facial expression.

Introducing a novel dataset for facial emotion recognition and demonstrating significant enhancements in deep learning performance through pre-processing techniques

Cognitive domains related to attention and executive functions (a set of cognitive processes that regulate, control, and manage other cognitive abilities) seem to influence the recognition of facial expressions in people with Alzheimer's disease (AD). We examined the relationship between facial expression recognition, global cognition and executive function in people with AD according to their cognitive level. In a cross-sectional design, we included 130 participants with AD divided into three groups based on their Mini-Mental State Examination (MMSE) scores: MMSE 1 (scores 23-28), MMSE 2 (scores 17-22), and MMSE 3 (scores 11-16). Facial expression recognition ability was analyzed using the Faces Test. Executive function was analyzed using the Trail Making Test (TMT), the Verbal Fluency Test (VFT), the Semantic Fluency Test (SFT), the Digit Span Forward (DSF) and Backward (DSB) tests, and the Clock Drawing Test (CDT). In MMSE 1 group difficulties in divided attention and cognitive flexibility impacted the accuracy of facial expression recognition. In the MMSE 2 group, facial expression recognition was related to impairment in working memory. In the MMSE 3 group, the impact on facial expression recognition was directly related to visuoconstructive abilities. We observed that the executive resources involved in each evaluated group differed in terms of facial recognition task performance efficacy. Interventions at stimulating executive and visuoconstructive abilities in people with AD may contribute to better preservation of facial expression recognition.

This paper examines the performance of different machine and deep learning algorithms in classifying colon histological images using different feature extraction methods. The relationship between the feature extraction methods and the selected machine learning methods to improve the classification accuracy is analyzed. Widely used methods like local binary patterns, histograms of oriented gradients, Gabor filter and Dobeshi wavelets are investigated for feature extraction from colon histological images. The features extracted by histogram of oriented gradients and Gabor filter methods were used as a single joint feature vector. And popular machine learning methods such as Support vector machine, Decision trees, Random forest, k-nearest neighbors and Naive Bayesian method were used to classify the selected images. The paper also investigates ensemble methods using gradient bousting and voting classifier as examples. The authors also focus on the study of convolutional neural networks as they are one of the main deep learning methods at the moment. The classification methods selected for analysis are compared in terms of classification accuracy and time taken for training and recognition. All pre-defined and adjustable parameters of both feature extraction methods and classification methods were personally selected by the authors as a result of experimental studies, which were conducted using a software tool created in the Python programming language on a set of LC25000 histological images. The software created is easily customizable and can be used in the future to investigate classification methods on other datasets.

Face recognition is a challenging issue in field of multi-science, main contents of research is how to make computer have ability of face recognition face recognition technology involved in a lot, which is a key feature extraction and classification method, this paper focuses on study of related theory. Non-negative matrix factorization trapped MF) algorithm and local non-negative matrix factorization (LNMF) algorithm is a feature extraction method based on local features, has been successfully used in face recognition NMF algorithm in face recognition rate is low, although LNMF algorithm to a certain extent, improve recognition rate, but its price is to increase number of iterations. In addition, two algorithms have failed to solve good nonlinear separable problems the kernel method combined with LNMF algorithm, kernel local non-negative matrix factorization (KLNMF) algorithm, first by a nonlinear transformation of original space to high-dimensional space, making samples linearly separable, and then use LNMF algorithm to extract face features. In classification part, paper presents decision rules of classification of their own, and design based on NMF subspace classifier. DOI : http://dx.doi.org/10.11591/telkomnika.v12i3.4356 Full Text: PDF

Facial expression is significant for face-to-face communication since it is one of our body language that increases data information during the communication. In recent surveys, it found that feature extraction methods have influenced on facial recognition directly and they can outperform if some irrelevant features are eliminated. Consequently, a procedure of facial expression recognition using graph-based features and artificial neural network is proposed in this thesis, and the procedure can be divided into 2 phases. For the first phase, fourteen points are manually located to create graph with edges connecting among such points, followed by computation of the Euclidean distances from those edges to define them as features for training in the next phase. For the next phase, Multilayer Perceptrons with back-propagation learning algorithm is implemented to recognize six basic emotions from the corresponding feature vectors. To evaluate the performance, Cohn-Kanade AU-Coded facial expression database is applied to the recognition system under various kinds of feature and training technique, and the experimental results have shown that MLP (Graph-based features) without validation and MLP (Graph-based feature) with cross validation can achieve the highest recognition rate (95.24%). Therefore, it has illustrated that the combination of graph-based features and artificial neural network is the efficient way for the facial expression recognition.

In this study, we present the performance of Support Vector Machine (SVM), Convolution Neural Network (CNN), and Artificial Neural Network (ANN) with Bag of Words (BoW), Histogram of Oriented Gradients (HOG), and Image Pixels (IP) for face recognition. SVM, CNN, and ANN are machine learning approaches and has been used for pattern recognition, especially in face recognition technology. BoW, HOG, and IP are being used for image feature extraction. The testing has been conducted from publicly available AT&T face database. Every individual subject consists of 10 images with different facial expression, different illumination and the dimensions of the images is unified as 92-by-112 pixels with PGM formats. SVM achieved recognition accuracy of 97.00%, 96.00%, and 98.00% with subsequently BoW, HOG, and IP. CNN achieved recognition accuracy of 94.00%, 99.00%, and 99.50% with subsequently BoW, HOG, and IP. ANN achieved recognition accuracy of 96.00%, 99.00%, and 99.50% with subsequently BoW, HOG, and IP. The experimental results show that the IP with ANN approach clearly outperformed the others approaches.

International audience