A New Facial Expression Recognition Technique using 2-D DCT and Neural Networks Based Decision Tree

Facial expression recognition plays a vital role in realizing a highly intelligent human-machine interface, and has recently attracted much attention. In this paper, we propose a new facial expression recognition method that utilizes the 2D DCT, k-means algorithm and vector matching. This technique is based on two main intuitive ideas: (i) complicated facial expression categories such as "anger" and "sadness", may be divided into several subcategories with different subfeature spaces where the recognition task can be performed with higher accuracy and (ii) the k-means algorithm may be used to cluster these subcategories. A new image database with five facial expressions (neutral, smile, anger, sadness, surprise) of 60 women was constructed using a computationally efficient projection-based technique. Experimental results using the new database and an existing one (60 men) reveal that the new technique outperforms the standard vector matching technique and two recently developed methods using fixed-size and constructive one-hidden-layer neural networks. The mean recognition rate can be as high as 95% for the two databases.

Facial expression is one way humans convey their emotional states. Accurate recognition of facial expressions is essential in perceptual human-computer interface, robotics and mimetic games. This paper presents a novel approach to facial expression recognition from static images that combines fixed and adaptive 2-D filters in a hierarchical structure. The fixed filters are used to extract primitive features. They are followed by the adaptive filters that are trained to extract more complex facial features. Both types of filters are non-linear and are based on the biological mechanism of shunting inhibition. The features are finally classified by a support vector machine. The proposed approach is evaluated on the JAFFE database with seven types of facial expressions: anger, disgust, fear, happiness, neutral, sadness and surprise. It achieves a classification rate of 96.7%, which compares favorably with several existing techniques for facial expression recognition tested on the same database.

Facial expressions are an ideal means of communicating one’s emotions or intentions to others. This overview will focus on human facial expression recognition as well as robotic facial expression generation. In the case of human facial expression recognition, both facial expression recognition on predefined datasets as well as in real-time will be covered. For robotic facial expression generation, hand-coded and automated methods i.e., facial expressions of a robot are generated by moving the features (eyes, mouth) of the robot by hand-coding or automatically using machine learning techniques, will also be covered. There are already plenty of studies that achieve high accuracy for emotion expression recognition on predefined datasets, but the accuracy for facial expression recognition in real-time is comparatively lower. In the case of expression generation in robots, while most of the robots are capable of making basic facial expressions, there are not many studies that enable robots to do so automatically. In this overview, state-of-the-art research in facial emotion expressions during human–robot interaction has been discussed leading to several possible directions for future research.

Every person expresses himself or herself with different facial expressions. Nowadays, recognition of facial expression is a popular topic for researchers due to the practical function in a wide variety of areas. The learning of human facial expressions is a tough and tricky domain in pattern research society. Each individual facial expression is produced by non-rigid object contortions, and these contortions are man-dependent. The main motto of this paper was to invent and apply a structure for the automatic recognition of human facial expression.

This paper presents a novel facial expression recognition (FER) technique based on support vector machine (SVM) for the FER. Here it is called the FERS technique. First, the FERS technique develops a face detection method that combines the Haar-like features method with the self-quotient image (SQI) filter. As a result, the FERS technique possesses better detection rate because the face detection method gets more accurate in locating face regions of an image. The main reason is that the SQI filter can overcome the insufficient light and shade light. Subsequently, three schemes, the angular radial transform (ART), the discrete cosine transform (DCT) and the Gabor filter (GF), are simultaneously employed in the design of the feature extraction for facial expression in the FERS technique. More specifically, they are employed in constructing a set of training patterns for the training of an SVM. The FERS technique then exploits the trained SVM to recognize the facial expression for a query face image. Finally, experimental results show that the recognition performance of the FERS technique can be better than that of other existing methods under consideration in the paper.

Boosted NNE collections for multicultural facial expression recognition

Facial expression recognition (FER) techniques can be widely used in human-computer interaction, intelligent robots, intelligent monitoring, and other domains. Currently, FER methods based on deep learning have become the mainstream schemes. However, these methods have some problems, such as a large number of parameters, difficulty in being applied to embedded processors, and the fact that recognition accuracy is affected by facial deflection. To solve the problem of a large number of parameters, we propose a DSC-DenseNet model, which improves the standard convolution in DenseNet to depthwise separable convolution (DSC). To solve the problem wherein face deflection affects the recognition effect, we propose a posture normalization model based on GAN: a GAN with two local discriminators (LD-GAN) that strengthen the discriminatory abilities of the expression-related local parts, such as the parts related to the eyes, eyebrows, mouth, and nose. These discriminators improve the model’s ability to retain facial expressions and evidently benefits FER. Quantitative and qualitative experimental results on the Fer2013 and KDEF datasets have consistently shown the superiority of our FER method when working with multi-pose face images.

Facial expression research on unconstrained spontaneous expressions has benefited from recent advances in feature extraction, dimensionality reduction, and classification techniques. While the facial action coding relies on temporal predictors, state-of-the-art facial expression recognition techniques have been slow to adapt to temporal methods. Further, despite strong evidence of sparsity in the visual cortex, few approaches to facial understanding utilize sparse representations. This paper proposes using a temporal detector based upon facial dynamics, a static expression detector based on sparse representations, and an advanced temporal-sparse fused emotion estimator. Our approach leverages techniques from both computer vision and human brain research to produce a state-of-the-art emotion estimator on unconstrained faces in natural conditions.

In order to better improve online teaching during the epidemic, teachers can adjust the teaching according to the students’ understanding. During the epidemic period, online teaching has become a basic way for teachers to teach. One big problem with online teaching is that the lecturers find it hard to see the students’ facial expressions. According to the study, the students’ facial expressions are essential for the instructor to understand the students’ understanding of the course material. There are some relevant studies on the recognition of human facial expressions. However, most of them did not consider some expressions such as “Enlightened,” “Confused,” or “Bored.” This study helps teachers to improve the quality of teaching by designing a program to provide them with students’ facial expressions. The study, implemented on students’ computers, can capture their faces from a camera, identify some common and useful facial expressions, and send them to lecturers regularly during online lectures. The main research methods used in this paper are as follows: (1) Use the face recognition algorithm provided by OpenCV to capture faces. (2) Train a convolutional neural network model to recognize the expression “Happy,” “Surprised,” “Neutral,” “Enlightened,” “Confusion,” and “Boredom.” (3) Use the message protocol EMQX to transmit the expression information. This study can successfully capture faces, and about 80% accuracy can identify expressions and successfully transmit expression information. This study contains some expressions rarely studied by others and is innovative in the field of facial expression recognition. Datasets, deep learning models, and results are available for other research teams.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

The last two years have seen a rapid rise in the duration of time that both adults and children spend on screens, driven by the recent COVID-19 health pandemic. A key adverse effect is digital eye strain (DES). Recent trends in human-computer interaction and user experience have proposed voice or gesture-guided designs that present more effective and less intrusive automated solutions. These approaches inspired the design of a solution that uses facial expression recognition (FER) techniques to detect DES and autonomously adapt the application to enhance the user’s experience. This study sourced and adapted popular open FER datasets for DES studies, trained convolutional neural network models for DES expression recognition, and designed a self-adaptive solution as a proof of concept. Initial experimental results yielded a model with an accuracy of 77% and resulted in the adaptation of the user application based on the FER classification results. We also provide the developed application, model source code, and adapted dataset used for further improvements in the area. Future work should focus on detecting posture, ergonomics, or distance from the screen.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

Research on human facial expression recognition has become a growing field. One important step in the recognition of facial expressions is feature extraction. This research uses Geometric Moment Invariants (GMI) as a feature extraction method. Research on facial expression recognition using either the GMI method or another method use single face image as the dataset. Therefore, in this study uses GMI feature extraction to classify facial expressions on multiple face images. Face detection process uses Viola-Jones method on OpenCV and classification process uses Multi Class SVM method. The results are features for each expression and a small average accuracy of 5 times. Therefore, the classification is also done with the k-fold cross validation technique with another classification method. The average accuracy results are still small. It caused by the training image also using outer area of face in the image, so the background included as the image features. It is tested from k value 2 to10, and produce Multi Class SVM 10.2%, Decision Tree Classifier 14.73%, Random Forest Classifier 14.78%, Gaussian Naive Bayes 14.73%, Nearest Centroid 14.66%, MLP Classifier 11.09%, and Stochastic Gradient Descent Classifier 14.19%. The highest accuracy result is Random Forest Classifier method 14.78%. In Random Forest method, the best k value obtained is 4 with an average accuracy 16.18%.

Knowledge about people's emotions can serve as an important context for automatic service delivery in context-aware systems. Hence, human facial expression recognition (FER) has emerged as an important research area over the last two decades. To accurately recognize expressions, FER systems require automatic face detection followed by the extraction of robust features from important facial parts. Furthermore, the process should be less susceptible to the presence of noise, such as different lighting conditions and variations in facial characteristics of subjects. Accordingly, this work implements a robust FER system, capable of providing high recognition accuracy even in the presence of aforementioned variations. The system uses an unsupervised technique based on active contour model for automatic face detection and extraction. In this model, a combination of two energy functions: Chan---Vese energy and Bhattacharyya distance functions are employed to minimize the dissimilarities within a face and maximize the distance between the face and the background. Next, noise reduction is achieved by means of wavelet decomposition, followed by the extraction of facial movement features using optical flow. These features reflect facial muscle movements which signify static, dynamic, geometric, and appearance characteristics of facial expressions. Post-feature extraction, feature selection, is performed using Stepwise Linear Discriminant Analysis, which is more robust in contrast to previously employed feature selection methods for FER. Finally, expressions are recognized using trained HMM(s). To show the robustness of the proposed system, unlike most of the previous works, which were evaluated using a single dataset, performance of the proposed system is assessed in a large-scale experimentation using five publicly available different datasets. The weighted average recognition rate across these datasets indicates the success of employing the proposed system for FER.

In recent years, Facial Expression Recognition (FER) techniques have gained substantial attention within the realm of biometric technology due to their wide range of applications, including emotion analysis, human-computer interaction, and surveillance systems. This paper presents a robust and efficient FER system composed of three key steps. First, precise face detection is performed using the Viola-Jones algorithm, a well-established method for detecting facial features in real-time. Second, the detected images are enhanced using a Modified Contrast Limited Adaptive Histogram Equalization (M-CLAHE) technique to improve contrast and visibility. Finally, feature extraction and classification are carried out using three powerful Convolutional Neural Network (CNN) architectures: VGG16, ResNet50, and Inception-v3, all benefiting from Transfer Learning to boost performance. Experiments were conducted on two widely-used datasets, JAFEE and CK+, with Inception-v3 achieving remarkable accuracy, reaching 98.43% and 99.93% on the respective databases. These results underline the effectiveness and robustness of our approach, particularly through the use of Transfer Learning, which significantly enhances the overall performance of the model compared to existing methods.