“Mind Lift” – PersonalTherapist

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

This paper presents a discriminative temporal topic model (DTTM) for facial expression recognition. Our DTTM is developed by introducing temporal and categorical information into Latent Dirichlet Allocation (LDA) topic model. Temporal information is integrated by placing an asymmetric Dirichlet prior over document-topic distributions. The discriminative ability is improved by a supervised term weighting scheme. We describe the resulting DTTM in detail and show how it can be applied to facial expression recognition. Experiments on CMU expression database illustrate that the proposed DTTM is very effective in facial expression recognition.KeywordsFacial ExpressionTopic ModelLatent Dirichlet AllocationFacial Expression RecognitionActive Appearance ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Facial expression recognition (FER) in the wild is a challenging task due to some uncontrolled factors such as occlusion, illumination, and pose variation. The current methods perform well in controlled conditions. However, there are still two issues with the in-the-wild FER task: (i) insufficient descriptions of long-range dependency of expression features in the facial information space and (ii) not finely refining subtle inter-classes distinction from multiple expressions in the wild. To overcome the above issues, an end-to-end model for FER, named attention-modulated contextual spatial information network (ACSI-Net), is presented in this paper, with the manner of embedding coordinate attention (CA) modules into a contextual convolutional residual network (CoResNet). Firstly, CoResNet is constituted by arranging contextual convolution (CoConv) blocks of different levels to integrate facial expression features with long-range dependency, which generates a holistic representation of spatial information on facial expression. Then, the CA modules are inserted into different stages of CoResNet, at each of which the subtle information about facial expression acquired from CoConv blocks is first modulated by the corresponding CA module across channels and spatial locations and then flows into the next layer. Finally, to highlight facial regions related to expression, a CA module located at the end of the whole network, which produces attentional masks to multiply by input feature maps, is utilized to focus on salient regions. Different from other models, the ACSI-Net is capable of exploring intrinsic dependencies between features and yielding a discriminative representation for facial expression classification. Extensive experimental results on AffectNet and RAF_DB datasets demonstrate its effectiveness and competitiveness compared to other FER methods.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

Facial expression recognition (FER) plays a vital role in image processing according to the widespread development of human interactive applications. In the past few years, various researchers have focused on FER for implementing it in different applications. The existing system suffers from various complexities such as low accuracy, computational cost, and poor recognition performances. In this article, we proposed a novel concept to recognize facial expressions. The proposed work comprises three sections that are pre-processing, feature extraction, and classification. The pre-processing techniques remove the unwanted data from the original image and enhance the crucial details for further processing. The Convolutional Neural Network (CNN) is used for feature extraction. But, it yields lower performance in terms of feature extraction due to the shortage of hyperparameter tuning. Hence, the Multi-strategy Gravitational Search Algorithm (M-GSA) is utilized to extract the facial expression features from the eyebrow movement, nose, chin, and lip corner of the facial images. The facial expressions are classified via the Support Vector Machine (SVM) classifier. In this work, the top five facial expressions such as surprise, sad, happy, fear, and angry with three facial expression datasets such as FER-2013 dataset, CK+ dataset, and JAFFE dataset. Ultimately, the proposed method demonstrates better classification accuracy and recognition rates than different kinds of state-of-art methods.

In order to solve the problems of low average gradient and long recognition time in traditional facial expression recognition method, a multi-scale detail enhancement method for facial dynamic expression recognition of sprint athletes is proposed. A principal component analysis method was used to establish the facial expression feature subspace of sprinters, to project and reduce the dimension of the facial dynamic expression feature vector of sprinters, and to obtain the low frequency information and high frequency information of the facial image of sprinters by bilateral filtering. The multi-scale details of expression are enhanced by using side suppression network model and improving image S curve. The feature vector of facial dynamic expression is input into support vector machine to recognise the facial dynamic expression of sprinter. Experimental results show that the average value of annoying gradient is about 98 and the shortest time s. is about 1.9.

Emotions are pivotal in the learning process, highlighting the importance of identifying students' emotional states within educational settings. While neural network models, particularly those rooted in deep learning, have demonstrated remarkable accuracy in detecting primary emotions like happiness, sadness, fear, disgust, and anger from facial expressions in videos, these emotions occur infrequently in learning environments. Conversely, cognitive emotions such as engagement, confusion, frustration, and boredom are significantly more prevalent, transpiring five times more frequently than basic emotions. However, unlike basic emotions which are relatively distinct, cognitive emotions present a subtler distinction, necessitating the utilization of more sophisticated models for accurate recognition. The proposed work presents an efficient Facial Expression Recognition (FER) model for monitoring the student engagement in a learning environment by considering their facial expressions like boredom, frustration, confusion and engagement. The proposed methodology includes certain pre-processing steps followed by facial expression recognition founded on Efficient-Net B3 CNN in which the learning parameters are optimized using Circle-Inspired Optimization Algorithm (CIOA). Finally, the post processing stage estimates the frame-wise group engagement level (GEL) of students based on certain expression labels. Based on the acquired results, it is noted that the suggested Efficient-Net B3 CNN-CIOA based FER model provides promising results in terms of accuracy by 99.5%, precision by 99.2%, recall by 99.5% and f1-score by 99.6%, when compared with some state-of-art facial expression recognition approaches. Also, the suggested approach computational complexity is very much less than the compared existing approaches.

Recognition of facial expression has many applications including human-computer interaction, human emotion analysis, personality development, cognitive science, health-care, virtual reality, image retrieval, etc. In this paper we propose a new method for recognition of facial expression using local region specific mean optical flow and local binary pattern feature descriptor with support vector machine classification. In general, facial expression recognition techniques divide the face into regular grid (holistic representation) and the facial features are extracted. However, in this paper we divide the face into domain specific local regions. At first a robust optical flow is utilized to get mean optical flow in different directions for each local region which considers both local statistic motion information and its spatial location. The features are used only from the key frames; which are detected based on maximal mean optical flow magnitude within a sequence w.r.t. neutral frame. Now, the region specific local binary pattern is extracted from key frame and concatenated with mean optical flow features. The performance of the proposed facial expression recognition system has been validated on CK+ facial expression dataset.

In order to achieve high recognition rate, most facial expression recognition (FER) methods generate sufficient labeled facial images based on generative adversarial networks (GAN) to train model. However, these methods do not estimate the facial pose before passing the images to the generator, which affects the quality of generated images. And mode collapse is prone to occur during the training process, leading to generate a single-style facial images. To solve these problems, a FER model is proposed based on pose conditioned dendritic convolution neural network (PCD-CNN) with pose and expression. Before passing the facial images to the generator, PCD-CNN was used to process facial images, effectively estimating the facial landmarks to detect face and disentangle the pose. In order to accelerate the training speed of the model, PCD-CNN was based on the ShuffleNet-v2 framework. Every landmark of facial image was modeled by a separate ShuffleNet-DeconvNet, maintaining better performance with fewer parameters. To solve the mode collapse during image generation, we theoretically analyzed the causes, and implemented mini-batch processing on the discriminator in the model and directly calculated the statistical characteristics of the mini-batch samples. Experiments were carried out on the Multi-PIE and BU-3DFE facial expression datasets. Compared with current advanced methods, our method achieves higher accuracy 93.08%, and the training process is more stable.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Background/Objectives: There is only limited research work is going on in the field of facial expression recognition on low resolution images. Mostly, all the images in the real world will be in low resolution and might also contain noise, so this study is to design a novel convolutional neural network model (FERConvNet), which can perform better on low resolution images. Methods: We proposed a model and then compared with state-of-art models on FER2013 dataset. There is no publicly available dataset, which contains low resolution images for facial expression recognition (Anger, Sad, Disgust, Happy, Surprise, Neutral, Fear), so we created a Low Resolution Facial Expression (LRFE) dataset, which contains more than 6000 images of seven types of facial expressions. The existing FER2013 dataset and LRFE dataset were used. These datasets were divided in the ratio 80:20 for training and testing and validation purpose. A HDM is proposed, which is a combination of Gaussian Filter, Bilateral Filter and Non local means denoising Filter. This hybrid denoising method helps us to increase the performance of the convolutional neural network. The proposed model was then compared with VGG16 and VGG19 models. Findings: The experimental results show that the proposed FERConvNet_HDM approach is effective than VGG16 and VGG19 in facial expression recognition on both FER2013 and LRFE dataset. The proposed FERConvNet_HDM approach achieved 85% accuracy on Fer2013 dataset, outperforming the VGG16 and VGG19 models, whose accuracies are 60% and 53% on Fer2013 dataset respectively. The same FERConvNet_HDM approach when applied on LRFE dataset achieved 95% accuracy. After analyzing the results, our FERConvNet_HDM approach performs better than VGG16 and VGG19 on both Fer2013 and LRFE dataset. Novelty/Applications: HDM with convolutional neural networks, helps in increasing the performance of convolutional neural networks in Facial expression recognition. Keywords: Facial expression recognition; facial emotion; convolutional neural network; deep learning; computer vision

This paper proposes a unified facial motion tracking and expression recognition framework for monocular video. For retrieving facial motion, an online weight adaptive statistical appearance method is embedded into the particle filtering strategy by using a deformable facial mesh model served as an intermediate to bring input images into correspondence by means of registration and deformation. For recognizing facial expression, facial animation and facial expression are estimated sequentially for fast and efficient applications, in which facial expression is recognized by static anatomical facial expression knowledge. In addition, facial animation and facial expression are simultaneously estimated for robust and precise applications, in which facial expression is recognized by fusing static and dynamic facial expression knowledge. Experiments demonstrate the high tracking robustness and accuracy as well as the high facial expression recognition score of the proposed framework.

Facial expressions convey nonverbal cues which play an important role in interpersonal relations, and are widely used in behavior interpretation of emotions, cognitive science, and social interactions. In this paper we analyze different ways of representing geometric feature and present a fully automatic facial expression recognition (FER) system using salient geometric features. In geometric feature-based FER approach, the first important step is to initialize and track dense set of facial points as the expression evolves over time in consecutive frames. In the proposed system, facial points are initialized using elastic bunch graph matching (EBGM) algorithm and tracking is performed using Kanade-Lucas-Tomaci (KLT) tracker. We extract geometric features from point, line and triangle composed of tracking results of facial points. The most discriminative line and triangle features are extracted using feature selective multi-class AdaBoost with the help of extreme learning machine (ELM) classification. Finally the geometric features for FER are extracted from the boosted line, and triangles composed of facial points. The recognition accuracy using features from point, line and triangle are analyzed independently. The performance of the proposed FER system is evaluated on three different data sets: namely CK+, MMI and MUG facial expression data sets.