Facial Expression Classification Using PCA and Hierarchical Radial Basis Function Network

An approximating neural model, called hierarchical radial basis function (HRBF) network, is presented here. This is a self-organizing (by growing) multiscale version of a radial basis function (RBF) network. It is constituted of hierarchical layers, each containing a Gaussian grid at a decreasing scale. The grids are not completely filled, but units are inserted only where the local error is over threshold. This guarantees a uniform residual error and the allocation of more units with smaller scales where the data contain higher frequencies. Only local operations, which do not require any iteration on the data, are required; this allows to construct the network in quasi-real time. Through harmonic analysis, it is demonstrated that, although a HRBF cannot be reduced to a traditional wavelet-based multiresolution analysis (MRA), it does employ Riesz bases and enjoys asymptotic approximation properties for a very large class of functions. HRBF networks have been extensively applied to the reconstruction of three-dimensional (3-D) models from noisy range data. The results illustrate their power in denoising the original data, obtaining an effective multiscale reconstruction of better quality than that obtained by MRA.

Automatic analysis of human facial expressions is one of the challenging problems in machine vision systems. It has many applications in human-computer interactions such as, social signal processing, social robots, deceit detection, interactive video and behavior monitoring. In this paper, we develop a new method for automatic facial expression recognition based on facial muscle anatomy and human face structure. The algorithm finds approximate location of effective facial muscles and extracts features by measuring skin texture in 11 local patches. Seven facial expressions, including neutral are being classified in this study using AdaBoost classifier and other classifiers on MMI databases. Experimental results show that analyzing skin texture from selected local patches gives accurate and efficient information in order to identify different facial expressions.

We describe a computer vision system for observing facial motion by using an optimal estimation optical flow method coupled with geometric, physical and motion-based dynamic models describing the facial structure. Our method produces a reliable parametric representation of the face's independent muscle action groups, as well as an accurate estimate of facial motion. Previous efforts at analysis of facial expression have been based on the facial action coding system (FACS), a representation developed in order to allow human psychologists to code expression from static pictures. To avoid use of this heuristic coding scheme, we have used our computer vision system to probabilistically characterize facial motion and muscle activation in an experimental population, thus deriving a new, more accurate, representation of human facial expressions that we call FACS+. Finally, we show how this method can be used for coding, analysis, interpretation, and recognition of facial expressions.

Facial expression analysis is one of the most essential tools for behavior interpretation and emotion modeling in Intelligent Human-Computer Interaction (HCI). Although humans can easily interpret facial emotions, computers have great difficulty doing so. Analyzing changes and deformations in the face is one of the methods through which machines can interpret facial expressions. However, maintaining great precision while being accurate, stable, and quick is still challenging in this field. To address this issue, this research presents an innovative and novel method to fully automatically extract critical features from a face during a facial expression. Various machine learning models are used on these features to analyze emotions. We used the optical flow algorithm to extract motion vectors divided into sections on the subject’s face. Finally, each section and its symmetric section were used to calculate a new vector. The final features produce a state-of-the-art accuracy of over 98% in emotion classification in the Extended Cohen-Kanade (CK+) facial expression dataset. Furthermore, we proposed an algorithm to filter the most important features with an SVM classifier and achieved an accuracy of over 97 % by only looking at 15% of the face area.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Facial Expression Detection (FED) is a vital application that helps computers perceive human behavior via facial expression analysis. In this research paper, we embark on a quest to make FED systems better using sophisticated algorithms and methodologies. We aim to make FED models more accurate and efficient in performance, thus leading to more empathetic and natural human-computer interactions. We explore the technical complexities of FED, learning cutting-edge techniques like convolutional neural networks and recurrent neural networks to decode facial signals and identify expressions with greater precision. We also touch on issues of dataset diversity, deployment in real-world settings, ethical implications, and user experience, aiming to design FED systems that are not only technologically advanced but also sensitive to the privacy of individuals and cultural differences. Through our work, we hope to lay the groundwork for more compassionate and inclusive human-computer interactions, ultimately leading to a future where technology complements our understanding of emotions and deepens human relationships.

Facial Expression Recognition (FER) is an important area in human computer interaction. FER has different applications such as analysis of student behaviour in virtual class room, driver mood detection, security systems, and medicine. The analysis of facial expressions is an interesting and exciting problem. Feature extraction plays important role in any FER system. Local Binary Pattern (LBP) and its variants are popular for feature extraction due to simplicity in computation and monotonic illumination invariant property. However, the performance of LBP is poor in the presence of noise. This work proposes a novel approach for feature extraction to improve the performance of the FER. In this approach, the LBP is calculated considering 4-neighbors and diagonal neighbours separately. Further, for affective feature description, the concept of adaptive window and averaging in radial directions is introduced. This approach reduces the length of the feature vector as well as immune to noise. Support Vector Machine (SVM) is considered for classification. Recognition rate and confusion matrix are used to assess the performance of the proposed algorithm. Extensive experimental results on JAFFE, CK, FERG and FEI face databases show significant improvement in recognition rate compared to the available techniques both in noise free and noisy conditions.

Facial expression recognition is one of the most important tasks in human–computer interaction, affective computing, computer vision, and related work. Feature additive pooling and progressive fine‐tuning of the convolutional neural network (CNN) for facial expression recognition in a static image are introduced. Network is proposed that partially employs the visual geometry group (VGG)‐face model pre‐trained on a VGG‐face dataset. The characteristics and distribution of the facial expression images in each database are biased according to the purpose of the publicly available facial database used. To alleviate this problem, a CNN model is developed that merges progressively fine‐tuned CNNs into a single network. Experiments were carried out to validate the presented method using facial expression images from the Cohn–Kanade +, Karolinska directed emotional face, and Japanese female facial expression databases, and cross‐database evaluation results show that the method is superior to state‐of‐the‐art methods.

In order to solve the problems of low average gradient and long recognition time in traditional facial expression recognition method, a multi-scale detail enhancement method for facial dynamic expression recognition of sprint athletes is proposed. A principal component analysis method was used to establish the facial expression feature subspace of sprinters, to project and reduce the dimension of the facial dynamic expression feature vector of sprinters, and to obtain the low frequency information and high frequency information of the facial image of sprinters by bilateral filtering. The multi-scale details of expression are enhanced by using side suppression network model and improving image S curve. The feature vector of facial dynamic expression is input into support vector machine to recognise the facial dynamic expression of sprinter. Experimental results show that the average value of annoying gradient is about 98 and the shortest time s. is about 1.9.

Evaluating acoustic comfort through facial expression analysis under different sound environments

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

Several researchers have attempted automatic recognition and tracking of facial expressions in video frames of neutral and expressive faces. But, it is very hard to detect a neutral face in practical applications because the distinction between facial movement and head movement is difficult to make. To solve this distinction problem, we tried to initialize the head position using the central point of each eye and the tip of the nose to extract facial movements. Based on the detected neutral faces, the changing processes of facial expressions can be recognized by means of detecting the reference face (neutral face) and an algorithm of the recognition for facial expressions. In short, we assume that this method of detecting neutral faces can be applied to the recognition of facial expressions.

This paper proposes a unified facial motion tracking and expression recognition framework for monocular video. For retrieving facial motion, an online weight adaptive statistical appearance method is embedded into the particle filtering strategy by using a deformable facial mesh model served as an intermediate to bring input images into correspondence by means of registration and deformation. For recognizing facial expression, facial animation and facial expression are estimated sequentially for fast and efficient applications, in which facial expression is recognized by static anatomical facial expression knowledge. In addition, facial animation and facial expression are simultaneously estimated for robust and precise applications, in which facial expression is recognized by fusing static and dynamic facial expression knowledge. Experiments demonstrate the high tracking robustness and accuracy as well as the high facial expression recognition score of the proposed framework.

Facial expression is one of the most powerful and direct mediums embedded in human beings to communicate with other individuals’ feelings and abilities. In recent years, many surveys have been carried on facial expression analysis. With developments in machine vision and artificial intelligence, facial expression recognition is considered a key technique of the developments in computer interaction of mankind and is applied in the natural interaction between human and computer, machine vision and psycho- medical therapy. In this paper, we have developed a new method to recognize facial expressions based on discovering differences of facial expressions, and consequently appointed a unique pattern to each single expression.by analyzing the image by means of a neighboring window on it, this recognition system is locally estimated. The features are extracted as binary local features; and according to changes in points of windows, facial points get a directional motion per each facial expression. Using pointy motion of all facial expressions and stablishing a ranking system, we delete additional motion points that decrease and increase, respectively, the ranking size and strenghth. Classification is provided according to the nearest neighbor. In the conclusion of the paper, the results obtained from the experiments on tatal data of Cohn-Kanade demonstrate that our proposed algorithm, compared to previous methods (hierarchical algorithm combined with several features and morphological methods as well as geometrical algorithms), has a better performance and higher reliability.