Robust Facial Expression Recognition via Sparse Representation and Multiple Gabor filters

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.

In order to solve the problems of low average gradient and long recognition time in traditional facial expression recognition method, a multi-scale detail enhancement method for facial dynamic expression recognition of sprint athletes is proposed. A principal component analysis method was used to establish the facial expression feature subspace of sprinters, to project and reduce the dimension of the facial dynamic expression feature vector of sprinters, and to obtain the low frequency information and high frequency information of the facial image of sprinters by bilateral filtering. The multi-scale details of expression are enhanced by using side suppression network model and improving image S curve. The feature vector of facial dynamic expression is input into support vector machine to recognise the facial dynamic expression of sprinter. Experimental results show that the average value of annoying gradient is about 98 and the shortest time s. is about 1.9.

In this paper, two novel methods for facial expression recognition in facial image sequences are presented. The user has to manually place some of Candide grid nodes to face landmarks depicted at the first frame of the image sequence under examination. The grid-tracking and deformation system used, based on deformable models, tracks the grid in consecutive video frames over time, as the facial expression evolves, until the frame that corresponds to the greatest facial expression intensity. The geometrical displacement of certain selected Candide nodes, defined as the difference of the node coordinates between the first and the greatest facial expression intensity frame, is used as an input to a novel multiclass Support Vector Machine (SVM) system of classifiers that are used to recognize either the six basic facial expressions or a set of chosen Facial Action Units (FAUs). The results on the Cohn-Kanade database show a recognition accuracy of 99.7% for facial expression recognition using the proposed multiclass SVMs and 95.1% for facial expression recognition based on FAU detection.

In this paper, we propose a novel method for facial expression recognition. The facial expression is extracted from human faces by an expression classifier that is learned from boosting Haar feature based look-up-table type weak classifiers. The expression recognition system consists of three modules, face detection, facial feature landmark extraction and facial expression recognition. The implemented system can automatically recognize seven expressions in real time that include anger, disgust, fear, happiness, neutral, sadness and surprise. Experimental results are reported to show its potential applications in human computer interaction.

In social robot development of interaction system robot, it is necessary to develop the fundamental function such as the robot perception. Due to the robot should correctly interpret a behavior or mental expression of the human. If the robot has a good emotional insight of the human, it is the advantage for the robot perception. In this paper, we implement the significant technique that take an advantage to the robot such as the human detection, face detection and recognition. Basically, these techniques could further enable the robot capability of intelligent empathy from the expression of human. We intensively study the vision method for facial expression recognition (FER) to understanding the human emotion and interacting by the robot expression in particular case. The robot interaction is based on the interested person that the robot can recognize with their emotional expression. We also experiment the system in term of face-to-face between robot and user with demonstrate using the head robot along with the result, such as the performance of the perception and the behavior expression of the robot.

Facial expressions play important role in human communication. The understanding of facial expression is a basic requirement in the development of next generation human computer interaction systems. Researches show that the intrinsic facial features always hide in low dimensional facial subspaces. This paper presents facial parts based facial expression recognition system with sparse representation classifier. Sparse representation classifier exploits sparse representation to select face features and classify facial expressions. The sparse solution is obtained by solving l¹ -norm minimization problem with constraint of linear combination equation. Experimental results show that sparse representation is efficient for facial expression recognition and sparse representation classifier obtain much higher recognition accuracies than other compared methods.

Facial expression is a very useful channel for intelligent human computer communication. In this paper we propose a hybrid method to recognize facial expression. Our main contributions in this study are: first, face region is detected by combing Adaboost, Skin color model and motion history image; second, feature points representing different facial expressions are separated using optical flow; third, a support vector machine classifier is used to classify these feature point's info (location, distance, angle); last, tests to explore the whole facial expression recognition process are conducted and the results are satisfactory.

Facial expression is an important channel for human communication and can be applied in many real applications. One critical step for facial expression recognition (FER) is to accurately extract emotional features. Current approaches on FER in static images have not fully considered and utilized the features of facial element and muscle movements, which represent static and dynamic, as well as geometric and appearance characteristics of facial expressions. This paper proposes an approach to solve this limitation using "salient” distance features, which are obtained by extracting patch-based 3D Gabor features, selecting the "salient” patches, and performing patch matching operations. The experimental results demonstrate high correct recognition rate (CRR), significant performance improvements due to the consideration of facial element and muscle movements, promising results under face registration errors, and fast processing time. Comparison with the state-of-the-art performance confirms that the proposed approach achieves the highest CRR on the JAFFE database and is among the top performers on the Cohn-Kanade (CK) database.

ABSTRACTHuman beings show multiple responses to various emotional states, like anger, disgust, surprise, happiness, sadness, and fear. Among various emotions, facial expressions are widely informative as they exhibit a person's intentions and character. Facial expression recognition is used in many applications, such as marketing, research, customer service, neuroscience, and psychology. Traditional unimodal methods for facial expression recognition are ineffective due to the scarcity of data. In this paper, the Residual Fused Neuro‐Fuzzy Network (RF‐NFN) is used for the recognition of facial expressions and detection of emotion type using video and Electroencephalogram (EEG) signals. Here, the video frame is allowed for pre‐processing done by Non‐Local Means (NLM) filtering. Then, pre‐processed video frames and input EEG signals are fed toward feature extraction, which is then followed by feature selection. Finally, facial expressions are recognized and the type of emotion is detected by RF‐NFN, which is designed by incorporation of Hybrid Cascade Neuro‐Fuzzy Network (Hybrid Cascade NFN) and Deep Residual Network (DRN). Moreover, the performance of the RF‐NFN model is validated by three performance measures that exhibited a maximum accuracy of 90.88%, precision of 91.77%, and recall of 94.57%.

Facial expression recognition is a rapidly growing area of research in computer vision due to its immense applications such as human-computer interaction, psychology and intelligent robotic systems etc. Various methods have been proposed in past two decades aiming to solve different issues in this aspect. Most of the researches used distinct databases for the evaluation of their techniques, so it is very challenging task to infer which one is superior. In this paper, we have implemented three methods of facial expression recognition named: Principal Component Analysis (PCA), Artificial Neural Network (ANN) and Local Binary Pattern (LBP) and evaluated their performances using JAFFE database. Keywords: Facial Expression Recognition, Emotion Recognition, JAFFE Dataset, PCA, LBP, ANN.

Facial expression recognition is a practical application in the field of artificial intelligence. This article focuses on the introduction of the principles of each facial expression recognition method based on deep learning in chronological order, as well as the advantages and disadvantages of each method. The methods of facial expression recognition based on deep learning introduced in this article are as follows: Li Dunyu [1] proposed a method of LBP and BP neural network to realize facial expression recognition. And then the convolutional neural network is widely used, Zhou Benjun [2] merged LBP and convolutional neural network to realize facial expression recognition; Liu [3] establish a model that combines a recurrent neural network with a 3D convolutional neural network, and added a timeline so that the machine can extract and recognize facial expressions in a video; Jung [4] proposed to combine the convolutional neural network and the deep neural network to form a new neural network DTAGN, by using this new network he completes facial expression recognition; Shi Xugan [5] proposed a model of combining deep belief network with multi-layer perceptron (MLP), and successfully applied it to the research of facial expression recognition. By utilizing joint training of feature points and multiple networks Xia Tian [6] finish the research; Zhang Feifei [7] and other scholars proposed multi-pose facial expression recognition based on generating an adversarial network.

In facial expression recognition applications, the images are corrupted with random noise, and this affects the classification accuracy. This article proposes a maximum and Minimum Response-based Gabor (MMRG) that can encode the facial texture more discriminatively and eliminate random noise. Two code images are produced from the available Gabor images. Then, after dividing the code images into grids, feature vectors are formed using histograms. A technique based on the bat algorithm is proposed for the optimization of the Gabor filter banks as Bat Algorithm-based Gabor Optimization (BAGO). The MMRG increases the efficiency of Gabor filter-based features by precisely distinguishing the texture frequencies. It also helps in reducing the dimensions of feature vector which is a major problem in Gabor filter-based feature extraction. Radial Basis Function-Extreme Learning Machine (RBF-ELM) classifier is used for a faster and accurate multi-classification. The proposed approach has been evaluated with six datasets namely, Japanese Female Facial Expression (JAFFE), Cohn Kanade (CK+), Multi-media Understanding Group (MUG), Static Facial Expressions in the Wild (SFEW), Oulu-Chinese Academy of Science, Institute of Automation (Oulu-CASIA) and Man–Machine Interaction (MMI) datasets to meet a classification accuracy of 97.2, 97.4, 95.4, 35.4, 87.4 and 82.3% for seven class emotion detection, which is high when compared to other state-of –the-art methods.

Facial expressions serve as a means of conveying human emotions and individual intentions. The ability to perceive and interpret facial emotions is a relatively effortless job for humans, although it poses significant challenges when attempting to replicate this capability using a computer. Facial expressions can be detected from static photos, video, webcam data, or real-time photographs. The primary focus of this study is to create the SMM Facial Expression dataset and to develop a Convolutional Neural Network (CNN) model for accurately recognizing and classifying facial expressions. This model utilizes End-to-end (E2E) networks to analyze individual frames or clusters of frames received from the camera. The analysis is conducted through various layers of convolutional and pooling operations. The proposed model is evaluated on two benchmark datasets, add long form Cohn-Kanade (CK+) and Facial Expression Recognition 2013 Dataset (FER2013) for facial expression recognition (FER). The obtained accuracy rates are 85.27% and 82.18% for CK+ and FER2013, respectively. This study demonstrates that the SMM Facial dataset is comparable in quality to previously benchmarked datasets, and the proposed model holds potential for real-time facial expression recognition.

This paper proposes a method of facial expression recognition based on Zernike moments and the minimum classification error (MCE) based hidden Markov model (HMM). In the feature extraction of face, the method of Zernike moments feature extraction based on local feature regions is adopted. First, eyes and mouths are segmented from the facial expression image and Zernike moments feature vectors of eyes and mouth are extracted. In the classification of expression, using MCE criteria is adopted to improve the performance of HMM. The results show that the performance of expression recognition system can be improved to some extent by using Zernike moments and MCE criterion.