Influence of shape and texture features on facial expression recognition

Texture characteristics among the landmark points reflected in human faces are important features in the recognition of facial expression. In this paper, we propose a salient feature based facial expression on texture characteristics. Detection of effective landmarks are performed by appearance based models and corresponding texture regions are extracted from face images. Local Binary Pattern (LBP) is used to compute the texture feature. Texture feature is used for normalizing texture signatures. In addition, we introduce stability indices corresponding to texture features which are exploited as features in the present scope. Moreover, statistical features such as moment, skewness, kurtosis and entropy are also supplemented to the feature set. These salient features together are used as an input to Nonlinear AutoRegressive with eXogenous (NARX) for recognition of the human facial expression. The Cohn-Kanade (CK+), Japanese Female Facial Expression (JAFFE), MMI and MUG benchmark databases are used to conduct the experiments and the results justify the effectiveness of the proposed procedure.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

In automatic plant classification, plant identification based on digital leaf images is a challenging task. This paper proposes a Moth-Flame Optimization (MFO) based Deep Belief Network (DBN) method for plant leaf recognition. Initially, a combination of texture and shape features is applied for extracting features from the preprocessed image. Further, for leaf classification, the MFO optimizes the DBN parameters to minimize error and are used as classifiers. The classifier has been applied to five different sets of mango leaf images and achieved an accuracy of 98.5%. The experimental result indicates that it is feasible to automatically classify plants by using multi-feature extraction of plant leaf images in combination with MFO-based DBN.

With the work and study pressure on people increasing and the importance of mental health problems on the rise, facial expression analysis plays an important role in mental health auxiliary diagnosis and treatment (MHADT). This study proposes a facial expression localization and recognition model that integrates Main Directional Mean Optical Flow (MDMO) with the Transformer architecture. It addresses the problems of insufficient generalization ability and limited temporal modeling ability of traditional methods in facial expression recognition. The study is based on the authoritative Audio/Visual Emotion Challenge 2019 Depression Detection Sub-challenge (AVEC2019 DDS) dataset in the mental health field, which contains 163 training samples, 56 validation samples, and 56 test samples. With Explicit Shape Regression, Local Binary Patterns, Mnemonic Descent Method, Convolutional Neural Network, and benchmark models as comparison objects, it systematically evaluates the model's performance in error, accuracy, and processing speed. The results show that the model achieves the best performance in both facial expression localization and recognition tasks. The validation set errors are 7.26 and 6.85, the localization accuracy reaches 89.6%, and the recognition accuracy reaches 88.9%, which is significantly better than other methods. At the same time, its image processing time is 55 milliseconds (ms) and 44ms, balancing high precision and real-time performance. The study indicates that the fusion of MDMO and Transformer can effectively capture the spatial and temporal features of facial expressions. Thus, this fusion method provides an efficient, stable, and scalable technical solution for emotion recognition in MHADT. This improves both the FER and localization accuracy and effect. Besides, it provides a novel reference in both approaches and application level for the intelligent development of mental health evaluation.

Plant disease is main reason of agricultural crops production losses. Leaf disease in plant occurs due to fungai, virus and bacterias. Image contains various important features which is used in classification. In this paper author initially detect disease then classify disease using extracted features. It takes five diseased leaves (Black rot, Black Measles, Leaf blight, Septoria leaf spot, Bacterial spot) and healthy leaf images then identify leaf is diseased or healthy then if leaf is diseased then classify type of disease. Color features mean, standard deviation, skewness and kurtosis computed then Region based shape feature calculated to identify the size of spots. Texture feature calculated using gray level co-occurrence matrix (GLCM) which identifies texture of image using distance and 45° angle variation in GLCM. Extracted features sent to trained feed forward neural network and classify diseased with color, shape and texture feature individually and combination of all features then observe that combination of color, shape and texture feature improve the performance of classification accuracy.

Several researchers have attempted automatic recognition and tracking of facial expressions in video frames of neutral and expressive faces. But, it is very hard to detect a neutral face in practical applications because the distinction between facial movement and head movement is difficult to make. To solve this distinction problem, we tried to initialize the head position using the central point of each eye and the tip of the nose to extract facial movements. Based on the detected neutral faces, the changing processes of facial expressions can be recognized by means of detecting the reference face (neutral face) and an algorithm of the recognition for facial expressions. In short, we assume that this method of detecting neutral faces can be applied to the recognition of facial expressions.

The recognition rate of person-independent facial expression is generally not high, which limits the practical application of facial expression recognition. Aiming at this problem, this paper analyzes the reasons for the low recognition rate of person-independent facial expression, and proposes a recognition algorithm of person-independent facial expression based on improved LBP (Local Binary Pattern) and HOSVD (Higher-Order Singular Value Decomposition). The algorithm has the following contributions of facial expression recognition framework. In the stage of facial expression feature extraction, the transient features extracted by LDP(Local Directional Pattern) and the persistent features extracted by CBP(Centralized Binary Pattern) are integrated to improve the discrimination of facial expression features. Moreover, in the stage of facial expression classification and recognition, the traditional nearest neighbor classification is changed into k-nearest neighbor pre-classification, and the regional energy calculated by HOSVD is used to determine the similarity of two images for secondary classification. Finally, in the extended Cohn-Kanade dataset and Oulu-CASIA NIR&VIS facial expression database, the theoretical analysis and experimental results show that the method has better recognition effect for solving the problem of person-independent facial expression recognition.

Facial emotion recognition (FER) is the technology or process of identifying and interpreting human emotions based on the analysis of facial expressions. It involves using computer algorithms and machine learning techniques to detect and classify emotional states from images or videos of human faces. Further, FER plays a vital role in recognizing and understanding human emotions to better interpret someone’s feelings, intentions, and attitudes. In the present time, it is widely used in various fields such as healthcare, human–computer interaction, law enforcement, security, and beyond. FER has a wide range of practical applications across various industries including Emotion Monitoring, Adaptive Learning, and Virtual Assistants. This paper presents a comparative analysis of FER algorithms, focusing on deep learning approaches. The performance of different datasets, including FER2013, JAFFE, AffectNet, and Cohn–Kanade, is evaluated using convolutional neural networks (CNNs), deep face, attentional convolutional networks (ACNs), and deep belief networks (DBNs). Among the tested algorithms, DBNs outperformed other algorithms, reaching the highest accuracy of 98.82%. These results emphasize the effectiveness of deep learning techniques, particularly DBNs, in FER. Additionally, outlining the advantages and disadvantages of current research on facial emotion identification might direct future research efforts in the direction of the most profitable directions.

In the past decades, the area of plastic-mulched cultivation has grown rapidly both on the global and regional scope, due to its remarkable efficiency of increasing crop production. However, the rapid expansion has brought a series of ecological and environmental problems in many regions, such as the vast amount of residues from the plastic-mulched farmland bring about aesthetic pollution and adverse environmental impacts on water, air and soil of the agricultural ecosystem and the terrestrial ecosystem. It is a serious problem that will do great damage to the sustainability of agriculture. In order to alleviate problems and to find solutions, the development of efficient methods to map plastic-mulched farmland as accurately as possible is of great significance to the policy-makers and scientists. In this paper, the optical remotely sensed data, Landsat-8 OLI, have been used in monitoring the plastic-mulched farmland in a study area located in Jizhou city, Hebei Province, China, where the plastic mulching is extensively practiced in farming and interlaced with uncovered farmland. The spectral and texture features of different ground objects have been statistically analyzed on the OLI imagery in the light of the training samples. Then the spectral and texture features that carry a certain degree of explanation for plastic-mulched farmland were selected for classification. The training and testing samples (bigger than 4 pixels × 4 pixels of panchromatic fused OLI imagery) were collected from Google Earth images, and were amended by comparing to false color composite Landsat bands 7 (SWIR), 5 (NIR) and 4 (RED). At the same time, the samples were purified according the J-M distance method. The Support Vector Machines (SVM) has been used as classifier to extract the plastic-mulched farmland based on the spectral features alone, on the texture features alone and on the combined spectral and texture features respectively. The classification result was validated using confusion matrix. The overall classification accuracy for plastic-mulched farmland based on spectra features alone, on texture features alone and on combined spectral and texture features are 89.45%, 94.68% and 94.73%, respectively. Their respective Kappa coefficients are 0.85, 0.92 and 0.92. And the producer's accuracy and user's accuracy of plastic-mulched farmland are 87.30% and 63.96% on spectral features, 83.03%, and 81.37% on texture features, 83.39% and 81.37% on the combined features, respectively. The commission and omission errors for plastic-mulched farmland are 36.04% and 12.70% on spectral features, 18.63% and 16.97% on texture features, 18.63% and 16.61% on the combined features. This study shows that the plastic-mulched farmland can be extracted effectively using SVM classifier based on the Landsat-8 imagery.

Four experiments are reported investigating recognition of emotional expressions in very briefly presented facial stimulus. The faces were backwardly masked by neutral facial displays and recognition of facial expressions was analyzed as a function of the manipulation of different parameters in the masking procedure. The main conclusion was that stimulus onset asynchrony between target and mask proved to be the principal factor influencing recognition of the masked expressions. In general, confident recognitions of facial expressions required about 100-150 msec, with shorter time for happy than for angry expressions. The manipulation of the duration of both the target and the mask, by itself, had only minimal effects.

A robust method is proposed in this paper to retrieve digital images using combination of color, texture and shape features. The proposed method consists of three stages. Shape detection is done based on TopHat transform to detect and crop object part of the image in the first stage. The second is included a texture feature extraction algorithm using color local binary patterns (CLBP) and local variance features together. Finally, to retrieve mostly closing matching images to the query, log likelihood ratio is used as non-similarity ratio. The performance of the proposed approach is evaluated using Corel and Simplicity image sets and it compared by some of other well-known approaches in terms of precision and recall which shows the superiority of the proposed approach. Shift invariant, Low noise sensitivity, rotation invariant are some of other advantages.

We proposed a facial motion tracking and expression recognition system based on video data. By a 3D deformable facial model, the online statistical model (OSM) and cylinder head model (CHM) were combined to track 3D facial motion in the framework of particle filtering. For facial expression recognition, a fast and efficient algorithm and a robust and precise algorithm were developed. With the first, facial animation and facial expression were retrieved sequentially. After that facial animation was obtained, facial expression was recognized by static facial expression knowledge learned from anatomical analysis. With the second, facial animation and facial expression were simultaneously retrieved to increase the reliability and robustness with noisy input data. Facial expression was recognized by fusing static and dynamic facial expression knowledge, the latter of which was learned by training a multi-class expressional Markov process using a video database. The experiments showed that facial motion tracking by OSM+CHM is more pose robust than that by OSM, and the facial expression score of the robust and precise algorithm is higher than those of other state-of-the-art facial expression recognition methods.

Facial expressions recognition and synthesis are important research fields to study how human beings reflect to environments in affective computing. With the rapid development of mathematical theory on multivariate statistics and multi-media technology especially image processing, facial expressions recognition researchers have achieved many useful results. Recently studies show that approaches to facial modeling and expressions recognition and synthesis analysis could be adapted to control security or even real-time health monitoring in the real world. Similarity, how to achieve facial expressions recognition and synthesis for independent-free mechanism is a central design in general. Based on the cognitive analysis of independent user's affective facial recognition researches we proposed an emotions composition model to dope out what are the user's new affective facial expressions. At this point, principal component, cluster and discriminate analysis were applied to show and verify independent user's affective expressions can be composited for synthesis by basic facial expressions such as happy, neutral and unhappy. Experiments were conducted to prove our models to be very significant.

Objectives: Attention-deficit/hyperactivity disorder (ADHD) is the most frequent neurobehavioural disorder in childhood. ADHD is associated with impaired academic performance, cognitive, and emotional deficits. Moreover, comorbid oppositional defiant disorder (ODD) is leading to more severe impairment in social performance. Social cognition involves recognition, encoding, and interpretation of emotions from faces. Basic facial expressions that include sadness, happiness, anger, disgust, fear, and surprise are the easiest emotions to recognize. We aimed to demonstrate facial expression recognition impairments that might occur more frequently in children with co-occurring ADHD/ODD than patients with ADHD only. Thus, children with the co-occurrence of ODD may suffer more severely from social and behavioural difficulties.Methods: Forty patients diagnosed with ADHD and/or co-occurring ADHD/ODD according to DSM-IV-TR criteria were compared with a parallel (by gender, age, and educational state) 11 healthy children as a control group in this study. Clear facial images of each emotion were used as well as two additional sets of photos include 50% blurred images and cropped eye images were added as distractors then all images represented with black and white tone for emotion recognition task via facial expression. Angry expressions presented as target expressions. DSM-IV-Based Screening and Rating Scale for Children and Adolescents with attention deficit and disruptive behaviour disorders, the Conners’ Teachers Rating Scale/Revised Long Form and the Conners’ Parent Rating Scale/Revised Long Form were used to provide diagnostic objectivity.Results: Control group statistically performed better than ADHD group on recognition of emotional facial expressions. Results showed no statistically significant differences between the ADHD and ADHD/ODD group on recognition of emotional facial expressions. However, according to results of emotion recognition task via facial expressions, there were statistically significant differences between pure ADHD and comorbid ADHD+ODD groups in happy and neutral expressions. ADHD/ODD group tend to attribute more meaning to neutral facial expressions. Additionally there was statistically significant difference between control group and ADHD group according to recognition of angry expressions. There were statistically significant differences between the groups according to recognition of sad expressions in all clear, blurred, and eye photographs.Conclusions: Difficulties in recognizing emotional facial expressions were observed in children with ADHD. A statistically significant association was established between presence of ADHD and impaired recognition of facial emotion expressions independent from the scores of the disruptive behaviour rating scale. Comorbid ODD was not associated with recognition of emotional facial expressions including angry expressions. Recognition of angry expressions was not found as a predictor for disruptive behaviour disorders.

Facial expression is significant for face-to-face communication since it is one of our body language that increases data information during the communication. In recent surveys, it found that feature extraction methods have influenced on facial recognition directly and they can outperform if some irrelevant features are eliminated. Consequently, a procedure of facial expression recognition using graph-based features and artificial neural network is proposed in this thesis, and the procedure can be divided into 2 phases. For the first phase, fourteen points are manually located to create graph with edges connecting among such points, followed by computation of the Euclidean distances from those edges to define them as features for training in the next phase. For the next phase, Multilayer Perceptrons with back-propagation learning algorithm is implemented to recognize six basic emotions from the corresponding feature vectors. To evaluate the performance, Cohn-Kanade AU-Coded facial expression database is applied to the recognition system under various kinds of feature and training technique, and the experimental results have shown that MLP (Graph-based features) without validation and MLP (Graph-based feature) with cross validation can achieve the highest recognition rate (95.24%). Therefore, it has illustrated that the combination of graph-based features and artificial neural network is the efficient way for the facial expression recognition.