Emotion Recognition from Occluded Facial Expressions Using Weber Local Descriptor

Facial emotion recognition using modified HOG and LBP features with deep stacked autoencoders

Biometrics is the study of unique characteristics present in the human body such as fingerprint, palm-print, retina, iris, footprint, etc. While other traits have been explored widely, only a few people have been considered the foot-palm region, despite having unique properties. Prior work has explored the foot shape features using length, width, major axis, minor axis, centroid, etc. but they are not reliable for personal verification due to similarity in the physical composition of two persons. It increases the demand for more unique features based on the footprint. Footprint texture features coming from creases of foot palm are unique and permanent like palmprint texture features. Hence the main objective of the paper is to investigate various kinds of texture feature techniques. These techniques will be further used in correct extraction of footprint features. After extraction of footprint features a detailed experimental analysis is performed to discover the uniqueness in foot texture. It is further utilized to test its viability as a human recognition trait. We describe a detailed feature extraction and classification technique applied to a collected footprint data-set. For feature extraction, we use three techniques: Gray Level Co-occurrence Matrix (GLCM), Histogram Oriented Gradient (HOG), and Local Binary Patterns (LBP). Feature classification is performed using four techniques: Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Ensemble Subspace Discriminant (ESD). GLCM provides less accuracy, while HOG generates a big feature vector which takes more execution time. LBP provides a trade-off between the accuracy and the execution time. Detailed quantitative experiments show: GLCM with LDA provides an accuracy of $$88.5\%$$ , HOG with Fine-KNN achieves $$86.5\%$$ accuracy and LBP with LDA achieves the accuracy of $$97.9\%$$ .

Facial expression which carries rich information of body behavior is the leading carrier of human affective and the symbol of intelligence. The main purpose of this paper is to recognize 3D human facial expression. The research in this paper includes the expression feature extraction algorithm and fusion with different kinds of feature. To contain more local texture feature information, we proposed a new feature of 3D facial expression named Local Threshold Binary Pattern (LTBP) which based on Local Binary Pattern (LBP). We calculate the difference of gray value standard between neighboring pixels and the center pixel as a threshold to binary instead of the traditional LBP operation which only comparison of size between neighboring pixels and the center pixel. After we get the LTBP feature, we fuse the LTBP and HOG (Histogram of Oriented Gradient) features to get multi-feature fusion for 3D facial expression recognition. Our algorithm of 3D facial expression recognition comprises three steps: (1) extracting two sets of feature vectors and establishing the correlation criterion function between the two sets of feature vectors; (2) solving the two sets canonical projective vectors and extracting their canonical correlation features by the framework of canonical correlation analysis algorithm; (3) doing feature fusion for classification by using proposed strategy. We have performed comprehensive experiments on the BU-3DFE database which is presently the largest available 3D face database. We have achieved verification rates of more than 90% for the 3D facial expression recognition.

<span lang="EN-US">In this paper, an experiment has been carried out based on a simple k-nearest neighbor (kNN) classifier to investigate the capabilities of three extracted facial features for the better recognition of facial emotions. The feature extraction techniques used are histogram of oriented gradient (HOG), Gabor, and local binary pattern (LBP). A comparison has been made using performance indices such as average recognition accuracy, overall recognition accuracy, precision, recall, kappa coefficient, and computation time. Two databases, i.e., Cohn-Kanade (CK+) and Japanese female facial expression (JAFFE) have been used here. Different training to testing data division ratios is explored to find out the best one from the performance point of view of the three extracted features, Gabor produced 94.8%, which is the best among all in terms of average accuracy though the computational time required is the highest. LBP showed 88.2% average accuracy with a computational time less than that of Gabor while HOG showed minimum average accuracy of 55.2% with the lowest computation time.</span>

Different styles of social interaction are one of the core characteristics of autism spectrum disorder (ASD). Social differences among individuals with ASD often include difficulty in discerning the emotions of neurotypical people based on their facial expressions. This review first covers the rich body of literature studying differences in facial emotion recognition (FER) in those with ASD, including behavioral studies and neurological findings. In particular, we highlight subtle emotion recognition and various factors related to inconsistent findings in behavioral studies of FER in ASD. Then, we discuss the dual problem of FER – namely facial emotion expression (FEE) or the production of facial expressions of emotion. Despite being less studied, social interaction involves both the ability to recognize emotions and to produce appropriate facial expressions. How others perceive facial expressions of emotion in those with ASD has remained an under-researched area. Finally, we propose a method for teaching FER [FER teaching hierarchy (FERTH)] based on recent research investigating FER in ASD, considering the use of posed vs. genuine emotions and static vs. dynamic stimuli. We also propose two possible teaching approaches: (1) a standard method of teaching progressively from simple drawings and cartoon characters to more complex audio-visual video clips of genuine human expressions of emotion with context clues or (2) teaching in a field of images that includes posed and genuine emotions to improve generalizability before progressing to more complex audio-visual stimuli. Lastly, we advocate for autism interventionists to use FER stimuli developed primarily for research purposes to facilitate the incorporation of well-controlled stimuli to teach FER and bridge the gap between intervention and research in this area.

Nowadays, researchers aim to enhance man-to-machine interactions by making advancements in several domains. Facial emotion recognition (FER) is one such domain in which researchers have made significant progresses. Features for FER can be extracted using several popular methods. However, there may be some redundant/irrelevant features in feature sets. In order to remove those redundant/irrelevant features that do not have any significant impact on classification process, we propose a feature selection (FS) technique called the supervised filter harmony search algorithm (SFHSA) based on cosine similarity and minimal-redundancy maximal-relevance (mRMR). Cosine similarity aims to remove similar features from feature vectors, whereas mRMR was used to determine the feasibility of the optimal feature subsets using Pearson’s correlation coefficient (PCC), which favors the features that have lower correlation values with other features—as well as higher correlation values with the facial expression classes. The algorithm was evaluated on two benchmark FER datasets, namely the Radboud faces database (RaFD) and the Japanese female facial expression (JAFFE). Five different state-of-the-art feature descriptors including uniform local binary pattern (uLBP), horizontal–vertical neighborhood local binary pattern (hvnLBP), Gabor filters, histogram of oriented gradients (HOG) and pyramidal HOG (PHOG) were considered for FS. Obtained results signify that our technique effectively optimized the feature vectors and made notable improvements in overall classification accuracy.

The ability to recognize emotion is important to wellbeing and building relationships with others, making this skill important in adolescence. Research investigating adolescents’ ability to recognize facial and vocal emotion expressions has reported differing conclusions about the pattern of emotion recognition across this developmental period. This systematic review aimed to clarify the pattern of recognition for facial and vocal emotion expressions, and the relationship of performance to different task and emotion expression characteristics. A comprehensive and systematic search of the literature was conducted using six databases. To be eligible for inclusion, studies had to report data for adolescents between 11 and 18 years of age and measure accuracy of the recognition of emotion cues in either the face or voice. A total of 2333 studies were identified and 47 met inclusion criteria. The majority of studies focused on facial emotion recognition. Overall, early, mid-, and late-adolescents showed a similar pattern of recognition for both facial and vocal emotion expressions with the exception of Sad facial expressions. Sex of the participant also had minimal impact on the overall recognition of different emotions. However, analysis showed considerable variability according to task and emotion expression characteristics. Future research needs to increase focus on recognition of complex emotions, and low-intensity emotion expressions as well as the influence of the inclusion of Neutral as a response option.

There have been quite a few studies on facial expression recognition over the years, and it is still a challenging subject due to the significant inter-class variability. Facial expression research in this field focuses on the development of techniques to identify, code, and extract facial expressions to improve prediction by computer. With great success of machine learning, the various texture descriptors are exploited to obtain a better performance. This paper proposes a method based on the aggregation between different descriptors Histogram of oriented Gradient (HOG) and Local Binary Pattern (LBP). First stage the input image has pre-processed to detect dace area which helps to extract most significant features. Then, Diagonal-HOG (D-HOG) also has extracted and aggregated all features. Finally, Support Vector Machine (SVM) has been used a classifier to classify each feature as well as aggregated features. We evaluate our method using Japanese Female Facial Expressions database (JAFFE), experimental results showed that the proposed method is accurate and efficient in recognizing facial expressions.

Current literature suggests that theory of mind (ToM) and recognition of facial emotion are impaired in behavioral variant frontotemporal dementia (bvFTD). In contrast, studies suggest that ToM is spared in Alzheimer disease (AD). However, there is controversy whether recognition of emotion in faces is impaired in AD. This study challenges the concepts that ToM is preserved in AD and that recognition of facial emotion is impaired in bvFTD. ToM, recognition of facial emotion, and identification of emotions associated with video vignettes were studied in bvFTD, AD, and normal controls. ToM was assessed using false-belief and visual perspective-taking tasks. Identification of facial emotion was tested using Ekman and Friesen's pictures of facial affect. After adjusting for relevant covariates, there were significant ToM deficits in bvFTD and AD compared with controls, whereas neither group was impaired in the identification of emotions associated with video vignettes. There was borderline impairment in recognizing angry faces in bvFTD. Patients with AD showed significant deficits on false belief and visual perspective taking, and bvFTD patients were impaired on second-order false belief. We report novel findings challenging the concepts that ToM is spared in AD and that recognition of facial emotion is impaired in bvFTD.

Patients with amygdala lesions were found to be impaired in recognizing the fear emotion both from face and from music. In patients with Parkinson's disease (PD), impairment in recognition of emotions from facial expressions was reported for disgust, fear, sadness and anger, but no studies had yet investigated this population for the recognition of emotions from both face and music. The ability to recognize basic universal emotions (fear, happiness and sadness) from both face and music was investigated in 24 medicated patients with PD and 24 healthy controls. The patient group was tested for language (verbal fluency tasks), memory (digit and spatial span), executive functions (Similarities and Picture Completion subtests of the WAIS III, Brixton and Stroop tests), visual attention (Bells test), and fulfilled self-assessment tests for anxiety and depression. Results showed that the PD group was significantly impaired for recognition of both fear and sadness emotions from facial expressions, whereas their performance in recognition of emotions from musical excerpts was not different from that of the control group. The scores of fear and sadness recognition from faces were neither correlated to scores in tests for executive and cognitive functions, nor to scores in self-assessment scales. We attributed the observed dissociation to the modality (visual vs. auditory) of presentation and to the ecological value of the musical stimuli that we used. We discuss the relevance of our findings for the care of patients with PD.

Emotion recognition is becoming more and more important with the growing interests on intelligent and natural human-machine interaction. This paper presents a research on emotion recognition based on the combination of facial expression and human voices. The geometry shape of facial organs, represented by cubic spline coefficients, as well as the facial texture, represented by Histogram of Gradients (HOG), are combined together to describe the emotion shown on human face. A classifier for facial expression-based emotion recognition is trained using Support Vector Machine (SVM)with the image data provided by CHEAVD 2.0. Meanwhile, another voice-based emotion classifier is trained also using SVM with acoustic features extracted from the voices accompanying the video. The recognition results are then combined at decision-level using Bayesian rule. This approach can achieve an accuracy of 38.41 % with the test data of CHEAVD 2.0. More details of the testing result are shown in the paper. In the end, pros and constraints of our approach are discussed.

The face of man: Expressions of universal emotions in a New Guinea village

The authors examine facial emotion recognition and unirhinal olfactory performance in 19 schizophrenia patients and 14 comparison subjects. In patients, right nostril odor identification performance was positively related to overall emotion recognition accuracy, specifically, sad facial expressions. Olfactory and emotion recognition abilities appear significantly linked in schizophrenia.

In this study, we present the performance of Support Vector Machine (SVM), Convolution Neural Network (CNN), and Artificial Neural Network (ANN) with Bag of Words (BoW), Histogram of Oriented Gradients (HOG), and Image Pixels (IP) for face recognition. SVM, CNN, and ANN are machine learning approaches and has been used for pattern recognition, especially in face recognition technology. BoW, HOG, and IP are being used for image feature extraction. The testing has been conducted from publicly available AT&T face database. Every individual subject consists of 10 images with different facial expression, different illumination and the dimensions of the images is unified as 92-by-112 pixels with PGM formats. SVM achieved recognition accuracy of 97.00%, 96.00%, and 98.00% with subsequently BoW, HOG, and IP. CNN achieved recognition accuracy of 94.00%, 99.00%, and 99.50% with subsequently BoW, HOG, and IP. ANN achieved recognition accuracy of 96.00%, 99.00%, and 99.50% with subsequently BoW, HOG, and IP. The experimental results show that the IP with ANN approach clearly outperformed the others approaches.

Soft biometrie such as gender, age, and ethnicity detection grows quickly as biometrie system detection to support other solid biometric modality such as face, fingerprint, and iris. Nowadays, ethnicity detection plays an important role for pre-identify and re-identity people with different demographic background. The goal of this paper is ethnicity detection from ocular biometrics. In this work, two different level of fusion have been applied. The first level of fusion is feature fusion which fused two important local features which are Local Binary Pattern (LBP), and Histogram of Oriented Gradient (HOG). The second level of fusion have been made at the descion level for the best results of the classifiers. Moreover, four classifiers which are support vector machine (SVM), Multi-Layer Perceptron (MLP), Linear Discriminant Analysis (LDA), and Quadratic Discriminant Analysis (QDA) with overall ten kernels have been trained, validate, and tested on the FERET database to show the performance before and after fusion. The best result with an overall test performance accuracy of 98.5% using SVM with polynomial kernel that have been applied on the fused features of LBP and HOG.