Facial Recognition, Expression Recognition, and Gender Identification

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.

Cognitive domains related to attention and executive functions (a set of cognitive processes that regulate, control, and manage other cognitive abilities) seem to influence the recognition of facial expressions in people with Alzheimer's disease (AD). We examined the relationship between facial expression recognition, global cognition and executive function in people with AD according to their cognitive level. In a cross-sectional design, we included 130 participants with AD divided into three groups based on their Mini-Mental State Examination (MMSE) scores: MMSE 1 (scores 23-28), MMSE 2 (scores 17-22), and MMSE 3 (scores 11-16). Facial expression recognition ability was analyzed using the Faces Test. Executive function was analyzed using the Trail Making Test (TMT), the Verbal Fluency Test (VFT), the Semantic Fluency Test (SFT), the Digit Span Forward (DSF) and Backward (DSB) tests, and the Clock Drawing Test (CDT). In MMSE 1 group difficulties in divided attention and cognitive flexibility impacted the accuracy of facial expression recognition. In the MMSE 2 group, facial expression recognition was related to impairment in working memory. In the MMSE 3 group, the impact on facial expression recognition was directly related to visuoconstructive abilities. We observed that the executive resources involved in each evaluated group differed in terms of facial recognition task performance efficacy. Interventions at stimulating executive and visuoconstructive abilities in people with AD may contribute to better preservation of facial expression recognition.

These Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Automatic facial expression recognition system has many applications including, but not limited to, human behaviour understanding, detection of mental disorders, and synthetic human expressions. Recognition of facial expression by computer with high recognition rate is still a challenging task. Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classification. The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control

Introducing a novel dataset for facial emotion recognition and demonstrating significant enhancements in deep learning performance through pre-processing techniques

The ability to recognize facial expressions of emotion is vita l for effective social interaction. Facial expressions convey emotional cues (Adolphs, 2002, 2003; Darwin, 1872/1965; Ekman, 1992, 1993) and accurate recognition of these cues is a necessary step in the evaluation of interpersonal interactions and for the subsequent application of appropriate social skills (Collins & Nowicki, 2001; Hampson, van Anders, & Mullin, 2006; Philippot & Feldman, 1990). Research has shown support for the association between facial expression recognition (FER) abilities and social competency (Custrini & Feldman, 1989; Lancelot & Nowicki, 1997; Nowicki & Duke, 1994); relationship difficulties (Barth & Bastiani, 1997; Blair & Coles, 2000; Lancelot & Nowicki, 1997); and various psychological and psychiatric conditions (Blair et al., 2004; Penn & Combs, 2000; Russell, Chu & Phillips, 2006; Silver, Goodman, Knoll, & Isakov, 2004; van Beek & Dubas, 2008), including anxiety (Easter et al., 2005), bipolar disorder (Brotman et al., 2008), and psychopathology (Blair et al., 2004). Other studies have shown support for associations between FER deficits and childhood maltreatment (Pears & Fisher, 2005; Pollak, Cicchetti, Hornung, & Reed, 2000) and attachment (Cooley, 2005; Magai, Distel & Liker, 1995; Niedenthal, Brauer, Robin & Innes-Ker, 2002). According to emotions theorist Izard (2002), the inability to recognize nonverbal forms of emotion expression can negatively affect intra-and interpersonal behavior, and may serve as a risk factor for poor adjustment and future adverse outcomes. Although the particular mechanisms underlying FER are under debate, the study of the ability to recognize facial expressions of emotion continues to be the subject of research within various fields, including psychology (Hall, 2006; Ekman, Friesen, & Tomkins, 1971; Nowicki & Carton, 1992), psychiatry (Herba & Phillips, 2004; Wang, Lee, Sigman, & Dapretto, 2007), as well as computer science and philosophy (Adolphs, 2002). Facial expression recognition research findings have been provided from varied perspectives, such as neuroscience (Wang et al., 2007); social-cognitive processing and early childhood experiences (Masten et al., 2008; Mullins & Duke, 2004; & Pollack & Sinha, 2000); as well as perception (Pollak, Messner, Kisler, & Cohn, 2009). Other researchers have integrated information from differing fields, such as social psychology and perception (Calder, Young, Keane, & Dean, 2000; Pollak et al., 2009) and behavior studies and neuroscience (Herba & Phillips, 2004). Thus, facial recognition research appears to be bridging some of the natural gaps caused by distinct fields of investigation. The authors of this paper will attempt to present findings from FER studies that are particularly relevant for behavioral psychologists. Following a brief history of the study of FER, we highlight findings from various fields (e.g., social psychology, nonverbal processing), that focused on FER with particular clinical populations. Discussion of suggested clinical implications and applications from a clinical behavioral perspective, and suggestions for future research directions, concludes this article on the importance of FER in clinical work. History of Facial Expression Recognition Study Current research and public interest in facial expression recognition stems from a rich history. Scientific study and understanding of emotion is thought to have begun in the 19th century with Charles Darwin's The Expression of the Emotions in Man and Animals (originally published in 1872) and G.G. Duchenne de Bologne's The Mechanism of Human Facial Expression (originally published in 1862) (Mayne & Bonanno, 2001). These early works focused on the important role of facial displays in emotional life and introduced the theory that emotions may be understood as biologically-based reflex behaviors serving adaptive functions. …

The development of Facial Expression Recognition (FER) technology has significantly enhanced the naturalness and intuitiveness of human-robot interaction. In the field of service robots, particularly in applications such as production assistance, caregiving, and daily service communication, efficient FER capabilities are crucial. However, existing Convolutional Neural Network (CNN) models still have limitations in terms of feature representation and recognition accuracy for facial expressions. To address these challenges, we propose CAGNet, a novel network that combines multiscale feature aggregation and attention mechanisms. CAGNet employs a deep learning-based hierarchical convolutional architecture, enhancing the extraction of features at multiple scales through stacked convolutional layers. The network integrates the Convolutional Block Attention Module (CBAM) and Global Average Pooling (GAP) modules to optimize the capture of both local and global features. Additionally, Batch Normalization (BN) layers and Dropout techniques are incorporated to improve model stability and generalization. CAGNet was evaluated on two standard datasets, FER2013 and CK+, and the experiment results demonstrate that the network achieves accuracies of 71.52% and 97.97%, respectively, in FER. These results not only validate the effectiveness and superiority of our approach but also provide a new technical solution for FER. Furthermore, CAGNet offers robust support for the intelligent upgrade of service robots.

Facial expressions are an ideal means of communicating one’s emotions or intentions to others. This overview will focus on human facial expression recognition as well as robotic facial expression generation. In the case of human facial expression recognition, both facial expression recognition on predefined datasets as well as in real-time will be covered. For robotic facial expression generation, hand-coded and automated methods i.e., facial expressions of a robot are generated by moving the features (eyes, mouth) of the robot by hand-coding or automatically using machine learning techniques, will also be covered. There are already plenty of studies that achieve high accuracy for emotion expression recognition on predefined datasets, but the accuracy for facial expression recognition in real-time is comparatively lower. In the case of expression generation in robots, while most of the robots are capable of making basic facial expressions, there are not many studies that enable robots to do so automatically. In this overview, state-of-the-art research in facial emotion expressions during human–robot interaction has been discussed leading to several possible directions for future research.

Facial expression recognition has been widely used in human computer interaction (HCI) systems. Over the years, researchers have proposed different feature descriptors, implemented different classification methods, and carried out a number of experiments on various datasets for automatic facial expression recognition. However, most of them used 2D static images or 2D video sequences for the recognition task. The main limitations of 2D-based analysis are problems associated with variations in pose and illumination, which reduce the recognition accuracy. Therefore, an alternative way is to incorporate depth information acquired by 3D sensor, because it is invariant in both pose and illumination. In this paper, we present a two-stream convolutional neural network (CNN)-based facial expression recognition system and test it on our own RGB-D facial expression dataset collected by Microsoft Kinect for XBOX in unspontaneous scenarios since Kinect is an inexpensive and portable device to capture both RGB and depth information. Our fully annotated dataset includes seven expressions (i.e., neutral, sadness, disgust, fear, happiness, anger, and surprise) for 15 subjects (9 males and 6 females) aged from 20 to 25. The two individual CNNs are identical in architecture but do not share parameters. To combine the detection results produced by these two CNNs, we propose the late fusion approach. The experimental results demonstrate that the proposed two-stream network using RGB-D images is superior to that of using only RGB images or depth images.

Facial expression recognition (FER) represents one of the most prevalent forms of interpersonal communication, which contains rich emotional information. But it became even more challenging during the times of COVID, where face masks became a mandatory protection measure, leading to the challenge of occluded lower-face during facial expression recognition. In this study, deep convolutional neural network (DCNN) represents the core of both our full-face FER system and our masked face FER model. The focus was on incorporating knowledge distillation in transfer learning between a teacher model, which is the full-face FER DCNN, and the student model, which is the masked face FER DCNN via the combination of both the loss from the teacher soft-labels vs the student soft labels and the loss from the dataset hard-labels vs the student hard-labels. The teacher-student architecture used FER2013 and a masked customized version of FER2013 as datasets to generate an accuracy of 69% and 61% respectively. Therefore, the study proves that the process of knowledge distillation may be used as a way for transfer learning and enhancing accuracy as a regular DCNN model (student only) would result in 46% accuracy compared to our approach (61% accuracy).

Understanding how customers feel is one of the most valuable insights a business can gain. Traditionally, companies depend on surveys, ratings, or written feedbacks to collect reviews—but these methods are often time-consuming, biased, or ignored by users. With the rapid advancement of Artificial Intelligence (AI) and Machine Learning (ML), it is now possible to automatically detect and interpret human emotions using technologies like facial recognition, video analysis, and Natural Language Processing (NLP). This paper presents a comprehensive survey of the latest research efforts aimed at improving customer feedback systems using AI-powered emotion recognition. Many researchers try to understand emotions such as happiness, sad, anger, neutral, surprise by studying people’s facial expressions, often using images or videos taken from cameras or webcams. Popular techniques include Convolutional Neural Networks (CNNs) for image-based emotion classification, Support Vector Machines (SVMs) for quick categorization. This document gives formatting instructions for authors preparing papers for publication in the Proceedings of an IEEE conference. The authors must follow the instructions given in the document for the papers to be published. You can use this document as both an instruction set and as a template into which you can type your own text. Keywords: Emotion Recognition, Facial Expression Recognition, Machine Learning, CNN, Viola-Jones Algorithm

Facial expression recognition is a challenging problem in the scientific field of computer vision. Several face expression recognition (FER) algorithms are proposed in the field of machine learning, and deep learning to extract expression knowledge from facial representations. Even though numerous algorithms have been examined, several issues like lighting changes, rotations and occlusions. We present an efficient approach to enhance recognition accuracy in this study, advocates transfer learning to fine-tune the parameters of the pre-trained model (VGG19 model ) and non-pre-trained model convolutional neural networks (CNNs) for the task of image classification. The VGG19 network and convolutional network derive two channels of expression related characteristics from the facial grayscale images. The linear fusion algorithm calculates the class by taking an average of each classification decision on training samples of both channels. Final recognition is calculated using convolution neural network architecture followed by a softmax classifier. Seven basic facial emotions (BEs): happiness, surprise, anger, sadness, fear, disgust, and neutral facial expressions can be recognized by the proposed algorithm, The average accuracies for standard data set’s “CK+,” and “JAFFE,” 98.3 % and 92.4%, respectively. Using a deep network with one channel, the proposed algorithm can achieve well comparable performance.

Objectives: Attention-deficit/hyperactivity disorder (ADHD) is the most frequent neurobehavioural disorder in childhood. ADHD is associated with impaired academic performance, cognitive, and emotional deficits. Moreover, comorbid oppositional defiant disorder (ODD) is leading to more severe impairment in social performance. Social cognition involves recognition, encoding, and interpretation of emotions from faces. Basic facial expressions that include sadness, happiness, anger, disgust, fear, and surprise are the easiest emotions to recognize. We aimed to demonstrate facial expression recognition impairments that might occur more frequently in children with co-occurring ADHD/ODD than patients with ADHD only. Thus, children with the co-occurrence of ODD may suffer more severely from social and behavioural difficulties.Methods: Forty patients diagnosed with ADHD and/or co-occurring ADHD/ODD according to DSM-IV-TR criteria were compared with a parallel (by gender, age, and educational state) 11 healthy children as a control group in this study. Clear facial images of each emotion were used as well as two additional sets of photos include 50% blurred images and cropped eye images were added as distractors then all images represented with black and white tone for emotion recognition task via facial expression. Angry expressions presented as target expressions. DSM-IV-Based Screening and Rating Scale for Children and Adolescents with attention deficit and disruptive behaviour disorders, the Conners’ Teachers Rating Scale/Revised Long Form and the Conners’ Parent Rating Scale/Revised Long Form were used to provide diagnostic objectivity.Results: Control group statistically performed better than ADHD group on recognition of emotional facial expressions. Results showed no statistically significant differences between the ADHD and ADHD/ODD group on recognition of emotional facial expressions. However, according to results of emotion recognition task via facial expressions, there were statistically significant differences between pure ADHD and comorbid ADHD+ODD groups in happy and neutral expressions. ADHD/ODD group tend to attribute more meaning to neutral facial expressions. Additionally there was statistically significant difference between control group and ADHD group according to recognition of angry expressions. There were statistically significant differences between the groups according to recognition of sad expressions in all clear, blurred, and eye photographs.Conclusions: Difficulties in recognizing emotional facial expressions were observed in children with ADHD. A statistically significant association was established between presence of ADHD and impaired recognition of facial emotion expressions independent from the scores of the disruptive behaviour rating scale. Comorbid ODD was not associated with recognition of emotional facial expressions including angry expressions. Recognition of angry expressions was not found as a predictor for disruptive behaviour disorders.