Enhanced linear subspace methods for face and gait analysis

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

These Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Automatic facial expression recognition system has many applications including, but not limited to, human behaviour understanding, detection of mental disorders, and synthetic human expressions. Recognition of facial expression by computer with high recognition rate is still a challenging task. Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classification. The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

A human face has a distinct and unique characteristics which make it play a very critical role in recognizing facial expression in a “facial expression recognition system.” Identifying or as we say it detection of expressions plays a big and significant role in a facial expression recognition system. If we talk about a human being it becomes an easy task recognize expression in any particular image sequence, but at the same time if we talk about fully automated systems not many are currently available or capable to do so. The field of facial expression recognition do have many different applications and its importance, it might be used to have an interaction between a human being and a computer, here a user, without using his hands, can give commands or instruct to the computer system with the help of facial expression recognition system. Quite a few options are available to identify a face in an image in an efficient and accurate manner, although similar cannot be said for features detection in a video sequence frame. Most systems are still dependent on manual operations for same. Here in this paper we have emphasizes on color normalization and facial feature extraction which uses LBP (Local Binary Pattern) as an effective feature detection approach, where the existing algorithms have been modified to improve the facial expression recognition accuracy. The recognition accuracy on the Indian database is observe to be 94.7\%.

Face perception is an essential and significant problem in pattern recognition, concretely including Face Recognition (FR), Facial Expression Recognition (FER), and Race Categorization (RC). Though handcrafted features perform well on face images, Deep Convolutional Neural Networks (DCNNs) have brought new vitality to this field recently. Vanilla DCNNs are powerful at learning high-level semantic features, but are weak in capturing low-level image characteristic changes in illumination, intensity, and texture regarded as key traits in facial processing and feature extraction, which is alternatively the strength of human-designed feature descriptors. To integrate the best of both worlds, we proposed novel Random Pixel Difference Convolution (RPDC) which is efficient alternatives to vanilla convolutional layers in standard CNNs and can promote to extract discriminative and diverse facial features. By means of searched RPDC of high efficiency, we build S-RaPiDiNet, and achieve promising and extensive experiment results in FR (<inline-formula> <tex-math notation="LaTeX">$\approx 0.5$ </tex-math></inline-formula>&#x0025; improvement), FER (over 1&#x0025; growth), and RC (0.25&#x0025;&#x2013;3&#x0025; increase) than baseline network in vanilla convolution, showing strong generalization of RPDC.

Face recognition has many important applications in areas such as public surveillance and security, identity verification in the digital world, and modeling techniques in multimedia data management. Facial expression recognition is also important for targeted marketing, medical analysis, and human–robot interaction. In this paper, we survey a few techniques for facial analysis. We compare the cloud platform AWS Rekognition, convolutional neural networks, transfer learning from pre-trained neural nets, and traditional feature extraction using facial landmarks for this analysis. Although not comprehensive, this survey covers a lot of ground in the state-of-the-art solutions for facial analysis. We show that to get high accuracy, good-quality data and processing power must be provided in large quantities. We present the results of our experiments which have been conducted over six different public as well as proprietary image data sets.

Facial images are of critical importance in many real-world applications from gaming to surveillance. The current literature on facial image analysis, from face detection to face and facial expression recognition, are mainly performed in either RGB, Depth (D), or both of these modalities. But, such analyzes have rarely included Thermal (T) modality. This paper paves the way for performing such facial analyzes using synchronized RGB-D-T facial images by introducing a database of 51 persons including facial images of different rotations, illuminations, and expressions. Furthermore, a face recognition algorithm has been developed to use these images. The experimental results show that face recognition using such three modalities provides better results compared to face recognition in any of such modalities in most of the cases.

Subspace learning (SL) plays a key role in various learning tasks, especially those with a huge feature space. When processing multiple high-dimensional learning tasks simultaneously, it is of great importance to make use of the subspace extracted from some tasks to help learn others, so that the learning performance of all tasks can be enhanced together. To achieve this goal, it is crucial to answer the following question: How can the commonality among different learning tasks and, of equal importance, the individuality of each single learning task, be characterized and extracted from the given datasets, so as to benefit the subsequent learning, for example, classification? Existing multitask SL methods usually focused on the commonality among the given tasks, while neglecting the individuality of the learning tasks. In order to offer a more general and comprehensive framework for multitask SL, in this article, we propose a novel method dubbed commonality and individuality-based SL (CISL). First, we formally define the notions and objective functions of both commonality and individuality with respect to multiple SL tasks. Then, we design an iterative algorithm to solve the formulated objective functions, with the convergence of the algorithm being guaranteed. To show the generality of the proposed method, we theoretically analyze its connections to existing single-task and multitask SL methods. Finally, we demonstrate the necessity and effectiveness of incorporating both commonality and individuality by interpreting the learned subspaces and comparing the performance of CISL (in terms of the subsequent classification accuracy) with that of classical and state-of-the-art SL approaches on both synthetic and real-world multitask datasets. The empirical evaluation validates the effectiveness of the proposed method in characterizing the commonality and individuality for multitask SL.

Facial expression recognition (FER) has emerged as a pivotal interdisciplinary research domain that bridges computer science, psychology, neuroscience, and medicine. By mapping the FER scientific knowledge graph, this study aimed to explore the technological evolution and forecast future trends in this field. The study collected and cleaned the research on emotion perception in the Web of Science (WoS) database, and utilized the software CiteSpace (version 6.4R1) and R (BiblioShiny packages) software to create a scientific knowledge map. K-means was used for cluster analysis, and then the latent Dirichlet allocation (LDA) was employed to extract popular topics from the text of each cluster. Uniform manifold approximation and projection (UMAP) was utilized to reduce high-dimensional embeddings to a two-dimensional space. From a regional perspective, research is mainly distributed in countries or regions such as North America, Western Europe, East Asia, India, and Australia. Research on facial emotion recognition has focused primarily on neuroscience, psychiatry, and psychology. With the rapid development of computer technology, the interdisciplinary intersection is becoming increasingly important as FER has shown strong potential in identifying rare and neurological diseases. Furthermore, the evolution of artificial intelligence (AI) has transformed facial expression feature extraction from manual methodologies to machine learning-based approaches. The rapid development of computer algorithms and AI has greatly improved the accuracy and speed of facial emotion recognition. As a technology capable of detecting instantaneous emotional changes, FER holds promising prospects in fields such as neuroscience, emotion analysis, and pain assessment.

Facial Expression Recognition (FER), also known as Facial Emotion Recognition, is an active topic in computer vision and machine learning fields. This paper analyzes different feature extraction and classification methods to propose an efficient facial expression recognition system. We have studied several feature extraction methods, including Histogram of Oriented Gradients (HOG), face-encoding, and the features extracted by a VGG16 Network. For classification, different classical classifiers, including Support Vector Machines (SVM), Adaptive Boosting (AdaBoost), and Logistic Regression, are evaluated with these features. Besides, we have trained a ResNet50 model from scratch and also tuned a ResNet50 which is pre-trained on VGGFace2 dataset. Finally, a part-based ensemble classifier is also proposed by focusing on different parts of face images. The experimental results provided on FER-2013 Dataset show that the tuned model of ResNet50 with a complete image of face, achieves higher performance than the other methods.

Cognitive domains related to attention and executive functions (a set of cognitive processes that regulate, control, and manage other cognitive abilities) seem to influence the recognition of facial expressions in people with Alzheimer's disease (AD). We examined the relationship between facial expression recognition, global cognition and executive function in people with AD according to their cognitive level. In a cross-sectional design, we included 130 participants with AD divided into three groups based on their Mini-Mental State Examination (MMSE) scores: MMSE 1 (scores 23-28), MMSE 2 (scores 17-22), and MMSE 3 (scores 11-16). Facial expression recognition ability was analyzed using the Faces Test. Executive function was analyzed using the Trail Making Test (TMT), the Verbal Fluency Test (VFT), the Semantic Fluency Test (SFT), the Digit Span Forward (DSF) and Backward (DSB) tests, and the Clock Drawing Test (CDT). In MMSE 1 group difficulties in divided attention and cognitive flexibility impacted the accuracy of facial expression recognition. In the MMSE 2 group, facial expression recognition was related to impairment in working memory. In the MMSE 3 group, the impact on facial expression recognition was directly related to visuoconstructive abilities. We observed that the executive resources involved in each evaluated group differed in terms of facial recognition task performance efficacy. Interventions at stimulating executive and visuoconstructive abilities in people with AD may contribute to better preservation of facial expression recognition.

Principal Component Analysis and Linear Discriminant Analysis are of the most known subspace learning techniques. In this paper, a way for training set enrichment is proposed in order to improve the performance of the subspace learning techniques by exploiting the a-priori knowledge that many types of data are symmetric. Experiments on artificial, facial expression recognition, face recognition and object categorization databases denote the robustness of the proposed approach.

A comprehensive account of almost all the feature extraction methods used in current face recognition systems is presented. Specifically we have made distinction in the holistic and local feature extraction and differentiate them qualitatively as opposed to quantitatively. It is argued that a global feature representation should be preferred over a bag-of-feature approach. The problems in current feature extraction techniques and their reliance on a strict alignment is discussed. Finally we have introduced to use face-GLOH signatures that are invariant with respect to scale, translation and rotation and therefore do not require properly aligned images. The resulting dimensionality of the vector is also low as compared to other commonly used local features such as Gabor, LBP etc. and therefore learning based methods can also benefit from it. In a typical multi-view face recognition task, where it is assumed to have several examples of a subject available for training, we have shown in an extensive experimental setting the advantages and weaknesses of commonly used feature descriptions. Our results show that under more realistic assumptions, most of the classifiers failed on conventional features. While using the introduced face-GLOH-signature representation is relatively less affected by large in-class variations. This has been demonstrated by providing a fair performance comparison of several classifiers under more practical conditions such as misalignments, large number of subjects and large pose variations. An important conclusion is to be drawn from the results on FERET is that conventional multi-view face recognition cannot cope well with regards to large pose variations. Even using a large number of training examples in different poses for a subject do not suffice for a satisfactory recognition. In order to solve the problem where only one training example per subject is available, many recent methods propose to use image synthesis to generate a given subject at all other views and then perform a conventional multi-view recognition (Beymer and Poggio, 1995; Gross et al, 2004). Besides the fact that such synthesis techniques cause severe artefacts and thus cannot preserve the identity of an individual, a conventional classification cannot yield good recognition results, as has been shown in an extensive experimental setting. More sophisticated methods are therefore needed in order to address pose invariant face recognition. Large pose differences cause significant appearance variations that in general are larger than the appearance variation due to identity. One possible way of addressing this is to learn these variations across each pose, more specifically by fixing the pose and establishing a correspondence on how a person’s appearance changes under this pose one could reduce the in-class appearance variation significantly. In our very recent work (Sarfraz and Hellwich, 2009), we demonstrate the usefulness of face-GLOH signature in this direction.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Face alignment and face recognition are two fundamental problems in the facial analysis community. For face alignment, it forms the basis for the accurate face recognition, age estimation, and facial expression recognition. For face recognition, it has been widely applied in various practical scenarios such as access control system, massive surveillance, human computer interaction, etc.. There mainly exist two lines of works in these two fields, namely holistic face alignment and recognition, and partial face alignment and recognition. Numerous holistic face alignment and recognition works