Facial Expression Recognition Method Based on Cascade Convolution Neural Network

PurposeFacial expression provides abundant information for social interaction, and the analysis and utilization of facial expression data are playing a huge driving role in all areas of society. Facial expression data can reflect people's mental state. In health care, the analysis and processing of facial expression data can promote the improvement of people's health. This paper introduces several important public facial expression databases and describes the process of facial expression recognition. The standard facial expression database FER2013 and CK+ were used as the main training samples. At the same time, the facial expression image data of 16 Chinese children were collected as supplementary samples. With the help of VGG19 and Resnet18 algorithm models of deep convolution neural network, this paper studies and develops an information system for the diagnosis of autism by facial expression data.Design/methodology/approachThe facial expression data of the training samples are based on the standard expression database FER2013 and CK+. FER2013 and CK+ databases are a common facial expression data set, which is suitable for the research of facial expression recognition. On the basis of FER2013 and CK+ facial expression database, this paper uses the machine learning model support vector machine (SVM) and deep convolution neural network model CNN, VGG19 and Resnet18 to complete the facial expression recognition.FindingsIn this study, ten normal children and ten autistic patients were recruited to test the accuracy of the information system and the diagnostic effect of autism. After testing, the accuracy rate of facial expression recognition is 81.4 percent. This information system can easily identify autistic children. The feasibility of recognizing autism through facial expression is verified.Research limitations/implicationsThe CK+ facial expression database contains some adult facial expression images. In order to improve the accuracy of facial expression recognition for children, more facial expression data of children will be collected as training samples. Therefore, the recognition rate of the information system will be further improved.Originality/valueThis research uses facial expression data and the latest artificial intelligence technology, which is advanced in technology. The diagnostic accuracy of autism is higher than that of traditional systems, so this study is innovative. Research topics come from the actual needs of doctors, and the contents and methods of research have been discussed with doctors many times. The system can diagnose autism as early as possible, promote the early treatment and rehabilitation of patients, and then reduce the economic and mental burden of patients. Therefore, this information system has good social benefits and application value.

The recognition of facial expressions of emotion is adaptive for human social interaction, but the ability to do this and the manner in which it is achieved differs among individuals. Previous functional neuroimaging studies have demonstrated that some brain regions, such as the inferior frontal gyrus (IFG), are active during the response to emotional facial expressions in healthy participants, and lesion studies have demonstrated that damage to these structures impairs the recognition of facial expressions. However, it remains to be established whether individual differences in the structure of these regions could be associated with differences in the ability to recognize facial expressions. We investigated this issue using acquired structural magnetic resonance imaging, and assessed the performance of healthy adults with respect to recognition of the facial expressions of six basic emotions. The gray matter volume of the right IFG positively correlated with the total accuracy of facial expression recognition. This suggests that individual differences in the ability to recognize facial expressions are associated with differences in the structure of the right IFG. Furthermore, the mirror neuron activity of the IFG may be important for establishing efficient facial mimicry to facilitate emotion recognition.

Expression recognition is a sub-task of computer vision. The accuracy of expression recognition accuracy is an important research content. However, Because facial expressions have different complexity and diversity at different times, expression recognition has always been a research difficulty. Aiming at the problem of feature extraction in expression recognition, to improve the accuracy of expression recognition, this paper proposes an expression recognition algorithm that improves the Resnet18 network model. First, in the Resnet18 network, The main branch of the residual module is embedded with the SEnet attention module. The feature layer is extracted from the channel dimension. Through the adaptive global average pooling method, the feature layer is compressed, and only the channel dimension information is taken. Then two full connection layers are used to pay attention to the channel information. Finally, the original feature layer is weighted. This method improves the accuracy of image classification tasks, Secondly, the Swish activation function is used to prevent saturation when the gradient gradually approaches zero during slow training, improve the performance of the whole neural network, and improve the accuracy of the network in classification. The experimental results show that, after improvement, compared with the original Resnet18 network, the accuracy on the public data set RAF-DB is improved by 0.59%, reaching 83.64%. Compared with the relevant classification methods, the detection accuracy is effectively improved.

Facial expressions, as a vital conduit for human emotional expression, are among the most observable features of machines in the field of computer vision. Consequently, facial expression recognition holds broad potential for applications in artificial intelligence and health monitoring, among others. Given the diversity and complexity of expressions, the development of efficient and accurate models for expression recognition is of significant importance. This paper systematically reviews the foundational knowledge and related research in facial expression recognition, analyzing the application of current primary models in expression recognition. Employing a combination of literature review and experimental analysis, this study evaluates existing facial expression recognition algorithms. Special attention is given to advanced models based on Convolutional Neural Networks (CNNs), with a detailed comparison of their architectures and characteristics, analyzing their performance under various conditions. The paper concludes with a summary of the latest advancements in the field of facial expression recognition and proposes potential directions for future research.

With the development of computer vision, face detection has gained great attention and rapid development. In this paper, we study the traditional Adaboost algorithm and build a model based on convolutional neural network. Specifically, the model adopts the Inception V2 proposed by GooLeNet, which reduces the parameters in convolution and expands the perceptual field. Then, we propose a weak classifier based on limited eigenvalues to adaptively select good classifiers. Finally, to handle the differences in facial expressions of the same person, we propose the joint loss function. With the above methods, the accuracy of facial expression recognition is improved.

There are two problems in facial expression recognition: one is that single-channel network can only extract a single feature, the other is that the multi-channel network can extract a variety of features, but it increases the amount of computation. This paper proposes an expression recognition method based on image fusion and transfer learning. It combines the knowledge and experience of traditional methods with the computing power of deep learning methods. The method firstly locates and segments the face according to human experience of knowing things. The segmented area is the face, eyes and mouth which affect expression most. Then the features of the three perspectives are fused to form a three-channel picture containing the face, eye and mouth area. Finally, it is sent to the network which has preliminary experience in understanding things for transfer learning. Experimental results show that this method has achieved good results on CK + and JAFFE datasets with the average accuracy 100% and 98.55%. Compared with single-channel and multi-channel convolutional neural networks, the presented method can effectively improve the accuracy of facial expression recognition.

There has always been a relationship between various human emotions and facial expressions. Moreover, facial expressions are an important feature that can reflect emotions. In computer vision area, face expression recognition has become one of the active research points. In order to explore the relationship between facial expressions and human emotional, this paper proposed a double-channel weighted mixture convolutional neural network (WMDCNN). The network effectively improves the accuracy of face expression recognition (FER). We constructed a double-channel network using the VGG16 network and a improve AlexNet network. The WMDCNN network takes the facial feature image and the processed LBP feature image as the network input respectively, and then weighted mixes the feature vectors obtained at the output. The final prediction is derived. The WMDCNN network was evaluated on two publicly available datasets, CK+ and FER2013. From the experimental results, it can be seen that the model has good effectiveness and advancement in face expression recognition.KeywordsFacial expression recognitionConvolutional neural networkWeighted mixture

Deep convolution network based emotion analysis towards mental health care

Cognitive domains related to attention and executive functions (a set of cognitive processes that regulate, control, and manage other cognitive abilities) seem to influence the recognition of facial expressions in people with Alzheimer's disease (AD). We examined the relationship between facial expression recognition, global cognition and executive function in people with AD according to their cognitive level. In a cross-sectional design, we included 130 participants with AD divided into three groups based on their Mini-Mental State Examination (MMSE) scores: MMSE 1 (scores 23-28), MMSE 2 (scores 17-22), and MMSE 3 (scores 11-16). Facial expression recognition ability was analyzed using the Faces Test. Executive function was analyzed using the Trail Making Test (TMT), the Verbal Fluency Test (VFT), the Semantic Fluency Test (SFT), the Digit Span Forward (DSF) and Backward (DSB) tests, and the Clock Drawing Test (CDT). In MMSE 1 group difficulties in divided attention and cognitive flexibility impacted the accuracy of facial expression recognition. In the MMSE 2 group, facial expression recognition was related to impairment in working memory. In the MMSE 3 group, the impact on facial expression recognition was directly related to visuoconstructive abilities. We observed that the executive resources involved in each evaluated group differed in terms of facial recognition task performance efficacy. Interventions at stimulating executive and visuoconstructive abilities in people with AD may contribute to better preservation of facial expression recognition.

In order to improve the facial expression recognition accuracy in complex contexts and decrease the identification time, this paper presents an efficient expression recognition algorithm based on feature points of the facial landmarks. In practice, facial expressions mainly focus on a few parts of muscle activity, which provides valuable reference to infer human emotions and intentions. Facial expression is a powerful nonverbal way for human to transmit information and reveal emotion. In this paper, we focus on geometric positions of key parts of the face. Firstly, the face area is detected in a photo or video, then the key parts of the face is extracted and the position correction is performed. A set of key points is located using the relative position of the face. This process can not only effectively avoid the impact of the environment and the light on the sample, but also greatly improve the recognition of facial expressions. With the development of human-computer interaction, facial expression recognition has become a hot topic in the field of pattern recognition. After years of development, facial expression recognition has achieved some success such as HOG. In this paper, HOG feature extraction method of facial expression is used as a contrast. Experimental results show that the proposed algorithm can extract key information and achieves higher recognition accuracy.

Immediate results and late outcomes after stent implantation in saphenous vein graft lesions: The multicenter u.s. palmaz-schatz stent experience: S.C. Wong, D.S. Baim, R.A. Schatz, P.S. Teirstein, S.B. King III, R.C. Curry Jr., R.R. Heuser, S.G. Ellis, M.W. Cleman, P. Overlie, J.W. Hirshfeld, C.M. Walker, F. Litvack, D. Fish, J.A. Brinker, M. Buchbinder, S. Goldberg, Y.C. Chuang, M.B. Leon. Washington

Facial expression recognition (FER) is an important means for machines to perceive human emotions and interact with human beings. Most of the existing facial expression recognition methods only use a single convolutional neural network to extract the global features of the face. Some insignificant details and features with low frequency are easy to be ignored, and part of the facial features are lost. This paper proposes a facial expression recognition method based on multi branch structure, which extracts the global and detailed features of the face from the global and local aspects respectively, so as to make a more detailed representation of the facial expression and further improve the accuracy of facial expression recognition. Specifically, we first design a multi branch network, which takes Resnet-50 as the backbone network. The network structure after Conv Block3 is divided into three branches. The first branch is used to extract the global features of the face, and the second and third branches are used to cut the face into two parts and three parts after Conv Block5 to extract the detailed features of the face. Finally, the global features and detail features are fused in the full connection layer and input into the classifier for classification. The experimental results show that the accuracy of this method is 73.7%, which is 4% higher than that of traditional Resnet-50, which fully verifies the effectiveness of this method.

This study assessed whether presenting 3D face stimuli could facilitate children’s facial expression recognition. Seventy-one children aged between 3 and 6 participated in the study. Their task was to judge whether a face presented in each trial showed a happy or fearful expression. Half of the face stimuli were shown with 3D representations, whereas the other half of the images were shown as 2D pictures. We compared expression recognition under these conditions. The results showed that the use of 3D faces improved the speed of facial expression recognition in both boys and girls. Moreover, 3D faces improved boys’ recognition accuracy for fearful expressions. Since fear is the most difficult facial expression for children to recognize, the facilitation effect of 3D faces has important practical implications for children with difficulties in facial expression recognition. The potential benefits of 3D representation for other expressions also have implications for developing more realistic assessments of children’s expression recognition.

<p>The hypothesis about the holistic nature of the perception of expressive behavior is tested using the material of posed dynamic expressions of seven emotional states. The availability of information about facial and body expressions, as well as about the context of the situation (a holistic image of human behavior; an image with blurred face; an image with blurred body and context) serves as the independent variable. The influence of perception conditions on the accuracy of expression recognition is assessed. It is shown that the accuracy of assessing expressions of fear and boredom is significantly reduced when information is available only about facial expressions. On the contrary, the neutral state of the poser is perceived most accurately when the face is blurred. The expression of disappointment is recognized better in conditions of full information than when the face is blurred. No significant changes in the accuracy of assessing the expressions of happiness, disgust, and excitement were found when the perception conditions changed. Analysis of the structure of erroneous answers allowed us to identify the main patterns of mixing perceived emotions. In particular, it was shown that in the absence of information about body movements and the context of the situation, the expression of fear is perceived as “shame” or other states of negative valence. Probably, the key to understanding situational fear is knowledge of the presence of an object that evokes this emotion. Situational manifestations of excitement and interest are similar, and therefore it is quite difficult to reliably distinguish between these emotions.</p>

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.