Facial expression decomposition

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Objectives: Attention-deficit/hyperactivity disorder (ADHD) is the most frequent neurobehavioural disorder in childhood. ADHD is associated with impaired academic performance, cognitive, and emotional deficits. Moreover, comorbid oppositional defiant disorder (ODD) is leading to more severe impairment in social performance. Social cognition involves recognition, encoding, and interpretation of emotions from faces. Basic facial expressions that include sadness, happiness, anger, disgust, fear, and surprise are the easiest emotions to recognize. We aimed to demonstrate facial expression recognition impairments that might occur more frequently in children with co-occurring ADHD/ODD than patients with ADHD only. Thus, children with the co-occurrence of ODD may suffer more severely from social and behavioural difficulties.Methods: Forty patients diagnosed with ADHD and/or co-occurring ADHD/ODD according to DSM-IV-TR criteria were compared with a parallel (by gender, age, and educational state) 11 healthy children as a control group in this study. Clear facial images of each emotion were used as well as two additional sets of photos include 50% blurred images and cropped eye images were added as distractors then all images represented with black and white tone for emotion recognition task via facial expression. Angry expressions presented as target expressions. DSM-IV-Based Screening and Rating Scale for Children and Adolescents with attention deficit and disruptive behaviour disorders, the Conners’ Teachers Rating Scale/Revised Long Form and the Conners’ Parent Rating Scale/Revised Long Form were used to provide diagnostic objectivity.Results: Control group statistically performed better than ADHD group on recognition of emotional facial expressions. Results showed no statistically significant differences between the ADHD and ADHD/ODD group on recognition of emotional facial expressions. However, according to results of emotion recognition task via facial expressions, there were statistically significant differences between pure ADHD and comorbid ADHD+ODD groups in happy and neutral expressions. ADHD/ODD group tend to attribute more meaning to neutral facial expressions. Additionally there was statistically significant difference between control group and ADHD group according to recognition of angry expressions. There were statistically significant differences between the groups according to recognition of sad expressions in all clear, blurred, and eye photographs.Conclusions: Difficulties in recognizing emotional facial expressions were observed in children with ADHD. A statistically significant association was established between presence of ADHD and impaired recognition of facial emotion expressions independent from the scores of the disruptive behaviour rating scale. Comorbid ODD was not associated with recognition of emotional facial expressions including angry expressions. Recognition of angry expressions was not found as a predictor for disruptive behaviour disorders.

In order to improve the recognition of facial expression recognition rate, facial expression recognition algorithm is based on a LBP and SVM decision tree. First will facial expression image is converted to LBP characteristic spectrum using LBP algorithm, and then the LBP characteristic spectrum into LBP histogram feature sequence, finally completes the classification and recognition of facial expression by SVM decision tree algorithm, and prove the effectiveness of the proposed method in the recognition of facial expression database in JAFFE.

In this paper, Facial Expression Recognition (FER) system, aid in modeling facial expression space for facial expression recognition by facial characteristic points and Gini index. Facial expression information are mostly concentrated on facial expression information regions, so the, eyes, eyebrows, and mouth are extracted from the input image. When a face image is input the feature extraction is performed which help in detecting facial characteristic points. Facial animation parameters are calculated in order to recognize one of the six basic facial expressions. The proposed technique is applied to JAFFE database consisting 30 images, each having six basic facial expression images (neutral, happy, surprise, fear, sad and angry). For certain expressions like, surprise, fear and happy it gives the promising results with best recognition rate, however for expressions like, neutral, sad and angry the recognition rate is lower. The proposed technique is implemented in MATLAB version 7.8.0.347 (R2009a).

Facial expression recognition (FER) has been widely studied since it can be used for various applications. However, most of FER techniques focus on discriminating typical facial expressions such as six basic facial expressions. Spontaneous facial expressions are not limited to such typical ones because the intensity of a facial expression varies depending on the intensity of an emotion. In order to utilize FER for real-world applications, therefore, it is necessary to discriminate slight difference of facial expressions. In this paper, we propose an effective FER method to recognize spontaneous facial expressions using ensemble learning which combines a number of naive Bayes classifiers. In addition, a method to estimate the intensity of facial expression is also proposed by using the classification results of the classifiers. The effectiveness of these methods are evaluated through an FER experiment and an experiment to estimate the intensity of facial expressions using a data set including spontaneous facial expressions.

In this paper, a new method for extracting expression-independent face features based on HOSVD (Higher-Order Singular Value Decomposition) is proposed and used for face recognition. In the new method, it is assumed that a facial expression could be represented by the facial expressions in the training set. In addition, the expression with higher similarity to the expression of test person has higher probability to represent the expression of test person. Expression-similarity weighted expression feature, which is the optimal estimation based on Bayesian estimation theory and the assumption, is used to estimate the face feature of the test person. As a result, the estimated face feature can reduce the influence of expression caused by insufficient training data and becomes less expression-dependent, and can be more robust to new expressions. The proposed method has been applied to Japanese Female Facial Expression (JAFFE) database. Expression-independent experimental results show the superiority of proposed method over the existing methods in terms of recognition rate and accumulative recognition rate.

It is fascinating to hear about the research conducted on facial expression recognition algorithms and its significance in the era of intelligence, especially with the development of the internet and IoT. The use of deep learning method and neural network VGG to study facial expressions sounds promising. It is encouraging to know that the network structure was improved and pooled using Soft pool, thereby enhancing the extraction of facial expression features. The integration of an attention mechanism to improve the ability of the network to recognize facial expressions appears to be a valuable addition to the research. it is impressive that the recognition rate of the improved network on the face expression dataset RAF-DB reached 86.1%, exceeding that of the original VGG network on this data set. this outcome suggests that the changed network is effective for facial expression recognition. thank you for sharing this information.

Due to the insufficient feature values extracted by traditional CNN networks, existing facial expression recognition algorithms struggle to effectively cope with the diversity and complexity of human emotional expressions, as well as the influence of lighting conditions and environmental factors, this paper proposes a facial expression recognition algorithm that integrates semantic features and attention mechanisms. By embedding attention mechanisms into the U-Net network, more prominent facial expression features are extracted, and the improved U-Net structure network is integrated in each layer of Resnet34 to extract richer feature values. Utilizing each layer of ResNet34 to process features, combined with U-Net and ECA -Net to generate weights, and outputting the weights and feature values through a residual network, more significant facial expression features can be extracted, enhancing the robustness and generalization of the model. Experimental evidence shows that the algorithm achieves a recognition rate of 78.1% on the FER2013 facial expression dataset, indicating superior accuracy.

Facial expressions are an ideal means of communicating one’s emotions or intentions to others. This overview will focus on human facial expression recognition as well as robotic facial expression generation. In the case of human facial expression recognition, both facial expression recognition on predefined datasets as well as in real-time will be covered. For robotic facial expression generation, hand-coded and automated methods i.e., facial expressions of a robot are generated by moving the features (eyes, mouth) of the robot by hand-coding or automatically using machine learning techniques, will also be covered. There are already plenty of studies that achieve high accuracy for emotion expression recognition on predefined datasets, but the accuracy for facial expression recognition in real-time is comparatively lower. In the case of expression generation in robots, while most of the robots are capable of making basic facial expressions, there are not many studies that enable robots to do so automatically. In this overview, state-of-the-art research in facial emotion expressions during human–robot interaction has been discussed leading to several possible directions for future research.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

The ability to recognize facial expressions of emotion is vita l for effective social interaction. Facial expressions convey emotional cues (Adolphs, 2002, 2003; Darwin, 1872/1965; Ekman, 1992, 1993) and accurate recognition of these cues is a necessary step in the evaluation of interpersonal interactions and for the subsequent application of appropriate social skills (Collins & Nowicki, 2001; Hampson, van Anders, & Mullin, 2006; Philippot & Feldman, 1990). Research has shown support for the association between facial expression recognition (FER) abilities and social competency (Custrini & Feldman, 1989; Lancelot & Nowicki, 1997; Nowicki & Duke, 1994); relationship difficulties (Barth & Bastiani, 1997; Blair & Coles, 2000; Lancelot & Nowicki, 1997); and various psychological and psychiatric conditions (Blair et al., 2004; Penn & Combs, 2000; Russell, Chu & Phillips, 2006; Silver, Goodman, Knoll, & Isakov, 2004; van Beek & Dubas, 2008), including anxiety (Easter et al., 2005), bipolar disorder (Brotman et al., 2008), and psychopathology (Blair et al., 2004). Other studies have shown support for associations between FER deficits and childhood maltreatment (Pears & Fisher, 2005; Pollak, Cicchetti, Hornung, & Reed, 2000) and attachment (Cooley, 2005; Magai, Distel & Liker, 1995; Niedenthal, Brauer, Robin & Innes-Ker, 2002). According to emotions theorist Izard (2002), the inability to recognize nonverbal forms of emotion expression can negatively affect intra-and interpersonal behavior, and may serve as a risk factor for poor adjustment and future adverse outcomes. Although the particular mechanisms underlying FER are under debate, the study of the ability to recognize facial expressions of emotion continues to be the subject of research within various fields, including psychology (Hall, 2006; Ekman, Friesen, & Tomkins, 1971; Nowicki & Carton, 1992), psychiatry (Herba & Phillips, 2004; Wang, Lee, Sigman, & Dapretto, 2007), as well as computer science and philosophy (Adolphs, 2002). Facial expression recognition research findings have been provided from varied perspectives, such as neuroscience (Wang et al., 2007); social-cognitive processing and early childhood experiences (Masten et al., 2008; Mullins & Duke, 2004; & Pollack & Sinha, 2000); as well as perception (Pollak, Messner, Kisler, & Cohn, 2009). Other researchers have integrated information from differing fields, such as social psychology and perception (Calder, Young, Keane, & Dean, 2000; Pollak et al., 2009) and behavior studies and neuroscience (Herba & Phillips, 2004). Thus, facial recognition research appears to be bridging some of the natural gaps caused by distinct fields of investigation. The authors of this paper will attempt to present findings from FER studies that are particularly relevant for behavioral psychologists. Following a brief history of the study of FER, we highlight findings from various fields (e.g., social psychology, nonverbal processing), that focused on FER with particular clinical populations. Discussion of suggested clinical implications and applications from a clinical behavioral perspective, and suggestions for future research directions, concludes this article on the importance of FER in clinical work. History of Facial Expression Recognition Study Current research and public interest in facial expression recognition stems from a rich history. Scientific study and understanding of emotion is thought to have begun in the 19th century with Charles Darwin's The Expression of the Emotions in Man and Animals (originally published in 1872) and G.G. Duchenne de Bologne's The Mechanism of Human Facial Expression (originally published in 1862) (Mayne & Bonanno, 2001). These early works focused on the important role of facial displays in emotional life and introduced the theory that emotions may be understood as biologically-based reflex behaviors serving adaptive functions. …

This paper proposes a unified facial motion tracking and expression recognition framework for monocular video. For retrieving facial motion, an online weight adaptive statistical appearance method is embedded into the particle filtering strategy by using a deformable facial mesh model served as an intermediate to bring input images into correspondence by means of registration and deformation. For recognizing facial expression, facial animation and facial expression are estimated sequentially for fast and efficient applications, in which facial expression is recognized by static anatomical facial expression knowledge. In addition, facial animation and facial expression are simultaneously estimated for robust and precise applications, in which facial expression is recognized by fusing static and dynamic facial expression knowledge. Experiments demonstrate the high tracking robustness and accuracy as well as the high facial expression recognition score of the proposed framework.

Facial expression recognition algorithm using LGC based on horizontal and diagonal prior principle