Facial Expression Recognition Using AAMICPF

Recognition of facial expressions becomes significant issue between human and robot interactions. The purpose of this paper is to study the alignment and tracking of facial features with optical flow and component-based active appearance model, and then analyze fitted points to recognize facial expressions. Using this method with accurate analysis and tracking of facial features in robot or computer. Consequently robot or computer can easily recognize user's facial expressions and emotional variation, and then response properly. We apply some realtime techniques and Active Appearance Model (AAM) on the cameras. A high-quality AAM alignment results depend on apposite selections of initial positions. Nevertheless it takes a lot of time when we apply image pyramid to get precise results. In this paper, we introduce a new method to apply AAM fitting and further solve above problems. In our fitting plan, we apply partial AAM fitting separately on mouth and eyes. Therefore we could make more efficient facial features alignment and then it becomes able to implement tracking to real-world video and realtime alignment. To get more stable partial AAM, we use multi-level optical flow to determine initial positions of facial feature models. It is relative easier to analyze user's emotional information and get accurate positions of facial features for further application in real world environments by the algorithm we developed.

This paper proposes a novel natural facial expression recognition method that recognizes a sequence of dynamic facial expression images using the differential active appearance model (AAM) and k-NNS as follows. First, we use the differential-AAM features (DAFs) that are computed from the difference of the AAM parameters between an input face image and a reference face image. Second, we perform the manifold learning. Third, we recognize the facial expression of the input face image in the embedded feature space using sequence based k-NN, k-NNS. Since we use DAFs, we also propose an effective way of finding the neutral facial expression as kernel density approximation. Experimental results show that (1) the DAFs improves the facial expression recognition performance than the conventional AAM features by 20% and (2) the sequence-based k-nearest neighbors classifier provides a 95% of facial expression recognition performance on the facial expression database (FED06).

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Facial expression recognition of intercepted video sequences based on feature point movement trend and feature block texture variation

In order to extract the expression features of critically ill patients, and realize the computer intelligent nursing, an improved facial expression recognition method is proposed based on the of active appearance model, the support vector machine (SVM) for facial expression recognition is taken in research, and the face recognition model structure active appearance model is designed, and the attribute reduction algorithm of rough set affine transformation theory is introduced, and the invalid and redundant feature points are removed. The critically ill patient expressions are classified and recognized based on the support vector machine (SVM). The face image attitudes are adjusted, and the self-adaptive performance of facial expression recognition for the critical patient attitudes is improved. New method overcomes the effect of patient attitude to the recognition rate to a certain extent. The highest average recognition rate can be increased about 7%. The intelligent monitoring and nursing care of critically ill patients are realized with the computer vision effect. The nursing quality is enhanced, and it ensures the timely treatment of rescue.

Most present research into facial expression recognition focuses on the visible spectrum, which is sensitive to illumination change. In this paper, we focus on integrating thermal infrared data with visible spectrum images for spontaneous facial expression recognition. First, the active appearance model AAM parameters and three defined head motion features are extracted from visible spectrum images, and several thermal statistical features are extracted from infrared (IR) images. Second, feature selection is performed using the F-test statistic. Third, Bayesian networks BNs and support vector machines SVMs are proposed for both decision-level and feature-level fusion. Experiments on the natural visible and infrared facial expression (NVIE) spontaneous database show the effectiveness of the proposed methods, and demonstrate thermal IR images’ supplementary role for visible facial expression recognition.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

Facial expression recognition (FER) plays an important role in human–computer interaction. The recent years have witnessed an increasing trend of various approaches for the FER, but these approaches usually do not consider the effect of individual differences to the recognition result. When the face images change from neutral to a certain expression, the changing information constituted of the structural characteristics and the texture information can provide rich important clues not seen in either face image. Therefore it is believed to be of great importance for machine vision. This study proposes a novel FER algorithm by exploiting the structural characteristics and the texture information hiding in the image space. Firstly, the feature points are marked by an active appearance model. Secondly, three facial features, which are feature point distance ratio coefficient, connection angle ratio coefficient and skin deformation energy parameter, are proposed to eliminate the differences among the individuals. Finally, a radial basis function neural network is utilised as the classifier for the FER. Extensive experimental results on the Cohn–Kanade database and the Beihang University (BHU) facial expression database show the significant advantages of the proposed method over the existing ones.

In this work we automatically segment the left ventricle (LV) in cardiac MR images in the end-diastole (ED) and end-systole (ES) phases using a novel approach that combines statistical and deterministic deformable models. A 3D Active Appearance Model (AAM) is used to segment the ED phase. The AAM texture model is trained on radial samples from gradient magnitude images to make the fitting process faster and more discriminative. A trained ED-to-ES shape correspondence model is used to map a given ED shape to an ES shape. Once the AAM model converges to a shape in ED, the correspondence model is used to get an approximate ES shape. We segment the LV in the ES phase by first fitting a deformable superquadric to the AAM converged shape (in ED) using data range forces and then tracking the LV using image and data range forces (for the ES shape obtained from correspondence model). We test our approach by performing leave-one-out training on a 35 patient datasets. The data comprises 19 normal patients and 16 patients having heart abnormalities (cardiomyopathy and myocardial infarction). The composition makes it a challenging data collection with significant shape variation. The performance of our method is evaluated by measuring the mismatch between automatically segmented and expert delineated contours using the Mean Perpendicular Distance (MPD) and Dice metrics. The average MPD is 2.6mm for ED and 3.7mm for ES (error mostly towards the apex and base). The average Dice is 0.9 for ED and 0.8 for ES. These results show good potential for clinical use.

Face registration is a challenging problem due in part to the non-rigid nature of human faces. Active Appearance Models (AAMs) have been proposed as a useful technique for face registration in part because they can account for changes in shape. Fitting of AAMs to imagery is typically done using the Gauss-Newton method; however this approach is known to fail when either the initial shape estimate is to far off and/or the appearance model fails to direct search toward a good match. In this paper, we employ Evolution Strategies (ES) to search for a near optimal fit, i.e. set of model parameters, that relate an AAM to a novel face image. In addition, we dramatically reduce the dimensionality of the search problem by analytically determining the optimal texture parameters associated with any given shape. Experimental results reveal that the proposed approach outperforms the standard AAM parameter estimation technique and some of its variants. Further, the difference is statistically significant. Finally, some basic limitations of AAMs are identified using fitness landscape analysis.

Active Appearance Models (AAMs) are generative parametric models that have been successfully used in the past to model faces. Anecdotal evidence, however, suggests that the performance of an AAM built to model the variation in appearance of a single person across pose, illumination, and expression (a Person Specific AAM) is substantially better than the performance of an AAM built to model the variation in appearance of many faces, including unseen subjects not in the training set (a Generic AAM). In this paper, we present an empirical evaluation that shows that Person Specific AAMs are, as expected, both easier to build and more robust to fit than Generic AAMs. Moreover, we show that: (1) building a generic shape model is far easier than building a generic appearance model, and (2) the shape component is the main cause of the reduced fitting robustness of Generic AAMs. We then proceed to describe two refinements to Generic AAMs to improve their performance: (1) a refitting procedure to improve the quality of the ground-truth data used to build the AAM and (2) a new fitting algorithm. For both refinements we demonstrate dramatically improved fitting performance. Finally, we evaluate the effect of these improvements on a combined model construction and fitting task.

In this paper a framework for face recognition using Active Appearance Model is proposed. Active appearance model has the ability to synthesize near-photo realistic images of the class learned from the training set. The recognition algorithm works on those synthesized images and tries to match the nearest face in the training set to the synthesized image.

This paper presents an approach to recognising the gender and expression of face images by means of Active Appearance Models (AAM). Features extracted by a trained AAM are used to construct Support Vector Machine (SVM) classifiers for 4 elementary emotional states (happy, angry, sad, neutral). These classifiers are arranged into a cascade structure in order to optimise overall recognition performance. Furthermore, it is shown how performance can be further improved by first classifying the gender of the face images using an SVM trained in a similar manner. Both gender-specific expression classification and expression-specific gender classification cascades are considered, with the former yielding better recognition performance. We conclude that there are gender-specific differences in the appearance of facial expressions that can be exploited for automated recognition, and that cascades are an efficient and effective way of performing multi-class recognition of facial expressions. 1.

Regression-based Active Appearance Model initialization for facial feature tracking with missing frames

Facial expressions, as a vital conduit for human emotional expression, are among the most observable features of machines in the field of computer vision. Consequently, facial expression recognition holds broad potential for applications in artificial intelligence and health monitoring, among others. Given the diversity and complexity of expressions, the development of efficient and accurate models for expression recognition is of significant importance. This paper systematically reviews the foundational knowledge and related research in facial expression recognition, analyzing the application of current primary models in expression recognition. Employing a combination of literature review and experimental analysis, this study evaluates existing facial expression recognition algorithms. Special attention is given to advanced models based on Convolutional Neural Networks (CNNs), with a detailed comparison of their architectures and characteristics, analyzing their performance under various conditions. The paper concludes with a summary of the latest advancements in the field of facial expression recognition and proposes potential directions for future research.