Emotion Recognition from Facial Expressions Using a Genetic Algorithm to Feature Extraction

In order to overcome the problems of high recognition error rate and low recognition efficiency existing in the traditional method of emotional recognition of weightlifters, a method of emotional recognition of weightlifters based on a non-negative matrix decomposition algorithm is proposed. The database of emotion recognition matching criteria is established, and the mapping relationship among emotion, facial expression of far mobilisation and weight lifting posture is formed. The real-time motion images of weightlifters are collected, and the image classifier is designed by using the non-negative matrix decomposition algorithm. The real-time images are divided into facial expression images and whole body motion images, and the features of the two images are extracted respectively. The extracted results are matched with the results of the standard database, and the real-time emotion recognition results of weightlifters are obtained. The experimental results show that compared with the traditional emotion recognition method, the proposed method improves the accuracy of emotion recognition by 50%.

Human emotion recognition has emerged as a critical component in the development of intelligent systems capable of understanding and responding to human emotional states. This comprehensive review examines the latest novel deep learning methods for emotion recognition, encompassing advances in neural architectures, multimodal fusion techniques, and innovative approaches to feature extraction and classification. The field has witnessed remarkable progress through the integration of sophisticated deep learning models including convolutional neural networks (CNNs), long short-term memory networks (LSTMs), transformer architectures, and generative adversarial networks (GANs). Recent breakthroughs have demonstrated exceptional performance improvements, with some methods achieving over 99% accuracy on benchmark datasets such as DEAP, SEED, and MAHNOB-HCI. This review analyzes the evolution from traditional machine learning approaches to state-of-the-art deep learning methodologies, examining their effectiveness across various modalities including electroencephalography (EEG), facial expressions, speech, and physiological signals.The paper provides a systematic analysis of fusion strategies, architectural innovations, and performance metrics while identifying key challenges and future research directions in the rapidly evolving landscape of emotion recognition technology.

A survey of emotion recognition methods with emphasis on E-Learning environments

Humans have the ability to perceive and depict a wide range of emotions. There are various models that can recognize seven primary emotions from facial expressions (joyful, gloomy, annoyed, dreadful, wonder, antipathy, and impartial). This can be accomplished by observing various activities such as facial muscle movements, speech, hand gestures, and so forth. Automatic emotion recognition is a significant issue that has been a hotly debated research topic in recent years. At the moment, several research people have taken a component in inheriting or extra multimodal for higher understanding. This paper indicates a method for emotion recognition that makes use of 3 modalities: facial images, audio indicators, and text detection from FER and CK+, RAVDESS, and Twitter tweets datasets, respectively. The CNN model achieved 66.67 percent on the FER-2013 dataset of labeled headshots while on the CK+ dataset, 98.4 percent accuracy was obtained. Finally, diverse fusion strategies had been approached, and each of those fusion techniques gave distinctive results. This project is a step towards the sense of interaction between human emotional aspects and the growing technology that is the future of development in today’s world.

Emotional state recognition is an important component for efficient human-computer interaction. Most existing works address this problem using 2D features, but they are sensitive to head pose, clutter, and variations in lighting conditions. The general 3D based methods only consider geometric information for feature extraction. In this paper, we present a real 3D visual features based method for human emotion recognition. 3D geometric information plus colour/density information of the facial expressions are extracted by 3D Gabor library to construct visual feature vectors. The filter's scale, orientation, and shape of the library are specified according to the appearance patterns of the 3D facial expressions. An improved kernel canonical correlation analysis (IKCCA) algorithm is proposed for final decision. From training samples, the semantic ratings that describe the different facial expressions are computed by IKCCA to generate a seven dimensional semantic expression vector. It is applied for learning the correlation with different testing samples. According to this correlation, we estimate the associated expression vector and perform expression classification. From experiment results, our proposed method demonstrates impressive performance.

Emotion recognition has become an important research area for advanced human-robot interaction. Through recognizing facial expressions, a robot can interact with a person with a more friendly manner. In this paper, we proposed a bimodal emotion recognition system by combining image and speech information. A novel information fusion strategy is proposed to set proper weights to two feature modalities based on their recognition reliability. The fusion weights are determined by the distance between test data and the classification hyperplane and the standard deviation of training samples. After normalization using the mean distance between training samples and the hyperplane, the fusion weight is set to represent the classification reliability of individual modality. In the latter bimodal SVM classification, the recognition result with higher weight is selected. The complete procedure has been implemented in a DSP-based system to recognize five facial expressions on-line in real time. The experimental results show that a recognition rate of 86.9% is achieved, an improvement of 5% compared with using only image information.

In this work, we have proposed STLEV (Siamese Neural Network with Triplet Loss and Long Short-Term Memory for Emotion recognition in Videos) a method for emotion recognition in videos. This method is beneficial for cognitive Human-Computer interaction. Emotion recognition from video is a challenging task as only few frames of a video will contain relevant information for recognizing emotions. Also identifying the apex frame is challenging. Emotion recognition from videos involves extracting the features from the frames of a video and classifying the sequence of features. For feature extraction we have used Siamese Neural Network (SNN) which is metric based meta-learning model. SNN is trained with triplet loss. For classification, Long Shot-Term Memory (LSTM) model is used. In this work we have used spatial and temporal aspects of video. Spatial features are extracted using SNN and temporal features using LSTM. Deep learning models need large amount of data for good accuracy. We have addressed this problem by leveraging few shot learning aspects of metric meta-learning so that fewer samples are needed for training the model used for feature extraction. Experiments done using BU-4DFE dataset achieved an accuracy of 87.5% which demonstrates the efficiency of the method STLEV.

Emotion recognition from EEG based on multi-task learning with capsule network and attention mechanism

Human emotions analysis has been the focus of many studies, especially in the field of Affective Computing, and is important for many applications, e.g. human-computer intelligent interaction, stress analysis, interactive games, animations, etc. Solutions for automatic emotion analysis have also benefited from the development of deep learning approaches and the availability of vast amount of visual facial data on the internet. This paper proposes a novel method for human emotion recognition from a single RGB image. We construct a largescale dataset of facial videos (FaceVid), rich in facial dynamics, identities, expressions, appearance and 3D pose variations. We use this dataset to train a deep Convolutional Neural Network for estimating expression parameters of a 3D Morphable Model and combine it with an effective back-end emotion classifier. Our proposed framework runs at 50 frames per second and is capable of robustly estimating parameters of 3D expression variation and accurately recognizing facial expressions from in the-wild images. We present extensive experimental evaluation that shows that the proposed method outperforms the compared techniques in estimating the 3D expression parameters and achieves state-of-the-art performance in recognising the basic emotions from facial images, as well as recognising stress from facial videos.

Emotion recognition using cross-modal attention from EEG and facial expression

Humans have traditionally found it simple to identify emotions from facial expressions, but it is far more difficult for a computer system to do the same. The social signal processing subfield of emotion recognition from facial expression is used in a wide range of contexts, particularly for human-computer interaction. Automatic emotion recognition has been the subject of numerous studies, most of which use a machine learning methodology. The recognition of simple emotions like anger, happiness, contempt, fear, sadness, and surprise, however, continues to be a difficult topic in computer vision. Deep learning has recently drawn increased attention as a solution to a variety of practical issues, including emotion recognition. In this study, we improved the convolutional neural network technique to identify 7 fundamental emotions and evaluated several preprocessing techniques to demonstrate how they affected the CNN performance. This research focuses on improving facial features and expressions based on emotional recognition. By identifying or recognising facial expressions that elicit human responses, it is possible for computers to make more accurate predictions about a person's mental state and to provide more tailored responses. As a result, we examine how a deep learning technique that employs a convolutional neural network might improve the detection of emotions based on facial features (CNN). Multiple facial expressions are included in our dataset, which consists of about 32,298 photos for testing and training. The preprocessing system aids in removing noise from the input image, and the pretraining phase aids in revealing face detection after noise removal, including feature extraction. As a result, the existing paper generates the classification of multiple facial reactions like the seven emotions of the facial acting coding system (FACS) without using the optimization technique, but our proposed paper reveals the same seven emotions of the facial acting coding system.

Facial emotion recognition (FER) is a topic that has gained interest over the years for its role in bridging the gap between Human and Machine interactions. This study explores the potential of real time FER modelling, to be integrated in a closed loop system, to help in treatment of children suffering from Autism Spectrum Disorder (ASD). The aim of this study is to show the differences between implementing Traditional machine learning and Deep learning approaches for FER modelling. Two classification approaches were taken, the first approach was based on classic machine learning techniques using Histogram of Oriented Gradients (HOG) for feature extraction, with a k-Nearest Neighbor and a Support Vector Machine model as classifiers. The second approach uses Transfer Learning based on the popular “Alex Net” Neural Network architecture. The performance of the approaches was based on the accuracy of randomly selected validation sets after training on random training sets of the Oulu-CASIA database. The data analyzed shows that traditional machine learning methods are as effective as deep neural net models and are a good compromise between accuracy, extracted features, computational speed and costs.

Patients with amygdala lesions were found to be impaired in recognizing the fear emotion both from face and from music. In patients with Parkinson's disease (PD), impairment in recognition of emotions from facial expressions was reported for disgust, fear, sadness and anger, but no studies had yet investigated this population for the recognition of emotions from both face and music. The ability to recognize basic universal emotions (fear, happiness and sadness) from both face and music was investigated in 24 medicated patients with PD and 24 healthy controls. The patient group was tested for language (verbal fluency tasks), memory (digit and spatial span), executive functions (Similarities and Picture Completion subtests of the WAIS III, Brixton and Stroop tests), visual attention (Bells test), and fulfilled self-assessment tests for anxiety and depression. Results showed that the PD group was significantly impaired for recognition of both fear and sadness emotions from facial expressions, whereas their performance in recognition of emotions from musical excerpts was not different from that of the control group. The scores of fear and sadness recognition from faces were neither correlated to scores in tests for executive and cognitive functions, nor to scores in self-assessment scales. We attributed the observed dissociation to the modality (visual vs. auditory) of presentation and to the ecological value of the musical stimuli that we used. We discuss the relevance of our findings for the care of patients with PD.

In recent years, affective computing has become a research hotspot in the area of natural language processing and Chinese emotion recognition is an important constituent. This paper proposes a method of Chinese emotion recognition based on three-way decisions. Given the emotion dictionary constructed firstly, the grammatical information of sentences, topic features of texts and three-way decisions are integrated and applied into Chinese emotion recognition, thus realizing the multi-label emotion recognition of sentences in Chinese texts. The results of experiments show that the method of Chinese emotion recognition, based on three-way decisions, has achieved excellent results in the emotion recognition of Chinese sentences.

In order to overcome the problem that the accuracy and efficiency of Mandarin emotion recognition is low based on singal-modal, a tri-modal Mandarin emotion recognition method based on the feature-level fusion of three important modal of speech, facial expression and body gesture is proposed. Firstly, different emotional features are extracted from speech, facial expressions and body gestures, and then the three modal emotion features are fused by IAGA method. Finally, SVM classifier is used to construct prediction model and emotion recognition is completed on it. The CHEAVD Chinease multi-modal emotion dataset is utilized to evaluate our proposed methods. Compared with the traditional mono-modal and dual-modal emotion recognition, the experimental results show that the dual-modal fusion has higher emotion recognition rate than the mono-modal, and the tri-modal fusion has higher emotion recognition rate than the dual-modal, which verifies the effectiveness of the tri-modal Mandarin emotion recognition