ML-DCNNet: Multi-level Deep Convolutional Neural Network for Facial Expression Recognition and Intensity Estimation

Changing facial expressions is a natural and powerful way of conveying personal intention, expressing emotion and regulating interpersonal communication. Automatic estimation of human facial expression intensity is an important step in enhancing the capability of human-robot interfaces. In this research, we have developed a system which can automatically estimate the intensity of facial expression in real-time. Based on isometric feature mapping, the intensity of expression is extracted from training facial transition sequences. Then, intelligent models including cascade neural networks and support vector machines are applied to model the relationship between the trajectories of facial feature points and expression intensity level. We have implemented a vision system which can estimate the expression intensity of happiness, anger and sadness in real-time.

Temporal dissociation in the brain processing of pain and anger faces with different intensities of emotional expression

Emotion recognition through facial expression analysis is an emerging research in Artificial Intelligence which faces many challenges. The problem is the variation of facial expressions that displays human emotions. Humans can subjectively express the same emotions in various ways. To overcome the problem of ambiguity in emotion expression, a fuzzy approach is developed to analyze the facial components in determining the type of emotion. In this study, we proposed a framework for fuzzy emotion recognition as a representation of the psychologist knowledge. Three stages in the fuzzy emotion recognition were facial feature extraction with Active Appearance Model; Semantic-linguistic facial features extraction; fuzzy emotion recognition with Fuzzy Emotion Classification. System performance testing provided the best results on extended Cohn Kanade (CK+) facial expression dataset, with the accuracy of linguistic facial component recognition 0.98, and accuracy of fuzzy emotion recognition 0.90. Testing was also performed on custom-made Indonesian Mixed Emotion Dataset (IMED) which resulted in accuracy of 0.87. The fuzzy emotion recognition has a potential to be applied in various real problems such as virtual counseling, stress detection, lie detection, and e-commerce.

Human face detection is the most important process in applications such as video surveillance, human computer interface, face recognition, and image database management. Face detection algorithms have primary factors that decrease a detection ratio : variation by lighting effect, location and rotation, distance of object, complex background. Due to variations in illumination, background, visual angle and facial expressions, the problem of machine face detection is complex. Algorithms were discussed in several papers about face detection and face recognition. But we know that implementation of these algorithm is not easy. We propose a face detection algorithm for color images in the presence of varying lighting conditions as well as complex background. We use the YCbCr color space since it is widely used in video compression standard and multimedia streaming services. Our method detects skin regions over the entire image, and then generates face candidate based on the spatial arrangement of these skin patches. The algorithm constructs eye, mouth, nose, and boundary maps for verifying each face candidate.

Facial expressions are well known to change with age, but the quantitative properties of facial aging remain unclear. In the present study, we investigated the differences in the intensity of facial expressions between older (n = 56) and younger adults (n = 113). In laboratory experiments, the posed facial expressions of the participants were obtained based on six basic emotions and neutral facial expression stimuli, and the intensities of their faces were analyzed using a computer vision tool, OpenFace software. Our results showed that the older adults expressed strong expressions for some negative emotions and neutral faces. Furthermore, when making facial expressions, older adults used more face muscles than younger adults across the emotions. These results may help to understand the characteristics of facial expressions in aging and can provide empirical evidence for other fields regarding facial recognition.

This paper presents a discriminative temporal topic model (DTTM) for facial expression recognition. Our DTTM is developed by introducing temporal and categorical information into Latent Dirichlet Allocation (LDA) topic model. Temporal information is integrated by placing an asymmetric Dirichlet prior over document-topic distributions. The discriminative ability is improved by a supervised term weighting scheme. We describe the resulting DTTM in detail and show how it can be applied to facial expression recognition. Experiments on CMU expression database illustrate that the proposed DTTM is very effective in facial expression recognition.KeywordsFacial ExpressionTopic ModelLatent Dirichlet AllocationFacial Expression RecognitionActive Appearance ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

These Human facial expressions convey a lot of information visually rather than articulately. Facial expression recognition plays a crucial role in the area of human-machine interaction. Automatic facial expression recognition system has many applications including, but not limited to, human behaviour understanding, detection of mental disorders, and synthetic human expressions. Recognition of facial expression by computer with high recognition rate is still a challenging task. Two popular methods utilized mostly in the literature for the automatic FER systems are based on geometry and appearance. Facial Expression Recognition usually performed in four-stages consisting of pre-processing, face detection, feature extraction, and expression classification. The human face is an important part of an individual’s body and plays an important role in knowing the individual’s mood. The face is where a human expresses all his basic emotions. In the existing system, they examine the mental state manually by assessing them but have many disadvantages like we cannot predict any accurate solutions based on the assessment score because we might be not sure what kind of emotions the human user would be all time. To overcome this problem, a novel system is proposed to suggest an effective solution for predicting the mental state dynamically, we propose a hybrid architecture invoking facial based emotion sequence, PEN test and IQ test. By consistent monitoring of a human’s emotion and subjecting to PEN and IQ tests, the human’s mental state is routed. Combination of the above three techniques provides promising results for mental state and self-control

This research aims to develop artificial intelligence (AI) algorithms in the context of facial recognition with a focus on increasing accuracy in difficult environmental conditions. Although facial recognition technology has made great progress, challenges such as poor lighting, variations in facial expressions, and head rotation are still problems that must be overcome. The research methodology involved collecting a wide dataset covering a wide variety of faces under various environmental conditions. This data is then processed and its features are extracted using computer image processing techniques. Furthermore, several deep neural network architectures, such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), were developed, trained, and evaluated for face recognition tasks. The expected result is the development of an AI algorithm that is able to overcome challenges in facial recognition with higher accuracy than existing methods. In particular, significant improvements in facial recognition accuracy are expected especially under low lighting conditions and variations in facial expressions. This research has a major impact in a variety of security applications, such as border surveillance, building access control, and corporate security. With higher facial recognition accuracy, security risks can be significantly reduced, resulting in safer and more efficient security solutions. In conclusion, this research aims to bring innovation in facial recognition technology through advanced AI approaches, with the potential to improve security in various contexts.

Previous studies have shown deficits in the perception of static emotional facial expressions in individuals with autism spectrum disorders (ASD), but results are inconclusive. Possibly, using dynamic facial stimuli expressing emotions at different levels of intensities may produce more robust results, since these resemble the expression of emotions in daily life to a greater extent. 30 Young adolescents with high-functioning ASD (IQ>85) and 30 age- and intelligence-matched controls (ages between 12 and 15) performed the Emotion Recognition Task, in which morphs were presented on a computer screen, depicting facial expressions of the six basic emotions (happiness, disgust, fear, anger, surprise and sadness) at nine levels of emotional intensity (20–100%). The results showed no overall group difference on the ERT, apart from a slightly worse performance on the perception of the emotions fear (p<0.03) and disgust (p<0.05). No interaction was found between intensity level of the emotions and group. High-functioning individuals with ASD perform similar to matched controls on the perception of dynamic facial emotional expressions, even at low intensities of emotional expression. These findings are in agreement with other recent studies showing that emotion perception deficits in high-functioning ASD may be less pronounced than previously thought.

Facial expression recognition (FER) has been widely studied since it can be used for various applications. However, most of FER techniques focus on discriminating typical facial expressions such as six basic facial expressions. Spontaneous facial expressions are not limited to such typical ones because the intensity of a facial expression varies depending on the intensity of an emotion. In order to utilize FER for real-world applications, therefore, it is necessary to discriminate slight difference of facial expressions. In this paper, we propose an effective FER method to recognize spontaneous facial expressions using ensemble learning which combines a number of naive Bayes classifiers. In addition, a method to estimate the intensity of facial expression is also proposed by using the classification results of the classifiers. The effectiveness of these methods are evaluated through an FER experiment and an experiment to estimate the intensity of facial expressions using a data set including spontaneous facial expressions.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Compared to more conventional techniques such as fingerprint recognition, the use of biometric authentication systems in computer-based examinations offers several advantages. To overcome the disadvantages of fingerprint biometrics, such as high administrative costs, long authentication times, and accuracy issues, this paper proposes an innovative solution that leverages state-of-the- art facial recognition technology. Although biometric fingerprint systems are reliable to a certain extent, they present significant challenges in the context of computer-assisted examinations. The authentication process can be time-consuming and can result in delays and logistical problems. These systems can also be prone to errors, including false positives and false negatives, compromising the integrity of the investigation process. These limitations require research into more efficient and accurate biometric solutions. The developed system uses FaceNet and Multi- Task Cascaded Convolutional Neural Network (MTCNN) algorithms in achieving superior accuracy, efficiency, as well as security in verifying candidates' identities during exams. FaceNet excels at facial recognition by mapping faces in a compact Euclidean space, ensuring high accuracy even with slight variations in facial expressions, angles or lighting conditions. MTCNN increases the robustness of the system through precise face detection and alignment, which are critical for reliable performance. The results show that the facial recognition system outperforms traditional fingerprint-based methods. Accuracy is significantly improved, reducing misidentifications, while streamlining the authentication process, minimizing delays, and improving overall efficiency. The robustness of the system ensures consistent performance despite environmental fluctuations.

Previous neuroimaging and neuropsychological studies have investigated the neural substrates which mediate responses to fearful, disgusted and happy expressions. No previous studies have investigated the neural substrates which mediate responses to sad and angry expressions. Using functional neuroimaging, we tested two hypotheses. First, we tested whether the amygdala has a neural response to sad and/or angry facial expressions. Secondly, we tested whether the orbitofrontal cortex has a specific neural response to angry facial expressions. Volunteer subjects were scanned, using PET, while they performed a sex discrimination task involving static grey-scale images of faces expressing varying degrees of sadness and anger. We found that increasing intensity of sad facial expression was associated with enhanced activity in the left amygdala and right temporal pole. In addition, we found that increasing intensity of angry facial expression was associated with enhanced activity in the orbitofrontal and anterior cingulate cortex. We found no support for the suggestion that angry expressions generate a signal in the amygdala. The results provide evidence for dissociable, but interlocking, systems for the processing of distinct categories of negative facial expression.

In the current scenario face identification and recognition is an important technique in surveillance. The face is a necessary biometric in humans. Therefore face detection plays a major job in computer vision applications. Several face recognition and emotions classification approaches have been presented throughout the last few decades of research to improve the rate of face recognition for thermal pictures. However, in real-time, lighting conditions might change due to several factors, such as the different times of capture, weather, etc. Due to variations in lighting intensity, the performance of the facial expression recognition system is not good. This paper proposed a model for human thermal face detection and expression classification. Four main steps were involved in this research. Initially, the Difference of the Gaussian (DOG) filter is utilized to crop the input thermal images and then normalize the images using the median filter in pre-processing step. Then, Efficient Net is used for extracting features such as shape, location, and occurrences from thermal face images. After that, detect human faces utilized by the YOLOv4 technique to better emotions classification. Finally, classify the emotions on faces by using the DenseNet technique into seven emotions such as happy, sad, disgust, surprise, anger, fear, and neutral. The proposed method outperforms state-of-art techniques for face recognition on thermal pictures, and classifies the expressions, according to experimentations on the RGB-D-T database. The accuracy, precision, recall, and f1-score metrics will be utilized with the database to assess the efficacy of the proposed methodology. The proposed models achieve a high classification accuracy of 95.97% on the RGB-D-T database. Furthermore, the outcomes show good precision for various face recognition tasks.

Learning spatial weighting for facial expression analysis via constrained quadratic programming