Applying psychophysical reverse correlation to high dimensional natural stimuli

Emotion recognition through facial expression analysis is an emerging research in Artificial Intelligence which faces many challenges. The problem is the variation of facial expressions that displays human emotions. Humans can subjectively express the same emotions in various ways. To overcome the problem of ambiguity in emotion expression, a fuzzy approach is developed to analyze the facial components in determining the type of emotion. In this study, we proposed a framework for fuzzy emotion recognition as a representation of the psychologist knowledge. Three stages in the fuzzy emotion recognition were facial feature extraction with Active Appearance Model; Semantic-linguistic facial features extraction; fuzzy emotion recognition with Fuzzy Emotion Classification. System performance testing provided the best results on extended Cohn Kanade (CK+) facial expression dataset, with the accuracy of linguistic facial component recognition 0.98, and accuracy of fuzzy emotion recognition 0.90. Testing was also performed on custom-made Indonesian Mixed Emotion Dataset (IMED) which resulted in accuracy of 0.87. The fuzzy emotion recognition has a potential to be applied in various real problems such as virtual counseling, stress detection, lie detection, and e-commerce.

Objectives: Attention-deficit/hyperactivity disorder (ADHD) is the most frequent neurobehavioural disorder in childhood. ADHD is associated with impaired academic performance, cognitive, and emotional deficits. Moreover, comorbid oppositional defiant disorder (ODD) is leading to more severe impairment in social performance. Social cognition involves recognition, encoding, and interpretation of emotions from faces. Basic facial expressions that include sadness, happiness, anger, disgust, fear, and surprise are the easiest emotions to recognize. We aimed to demonstrate facial expression recognition impairments that might occur more frequently in children with co-occurring ADHD/ODD than patients with ADHD only. Thus, children with the co-occurrence of ODD may suffer more severely from social and behavioural difficulties.Methods: Forty patients diagnosed with ADHD and/or co-occurring ADHD/ODD according to DSM-IV-TR criteria were compared with a parallel (by gender, age, and educational state) 11 healthy children as a control group in this study. Clear facial images of each emotion were used as well as two additional sets of photos include 50% blurred images and cropped eye images were added as distractors then all images represented with black and white tone for emotion recognition task via facial expression. Angry expressions presented as target expressions. DSM-IV-Based Screening and Rating Scale for Children and Adolescents with attention deficit and disruptive behaviour disorders, the Conners’ Teachers Rating Scale/Revised Long Form and the Conners’ Parent Rating Scale/Revised Long Form were used to provide diagnostic objectivity.Results: Control group statistically performed better than ADHD group on recognition of emotional facial expressions. Results showed no statistically significant differences between the ADHD and ADHD/ODD group on recognition of emotional facial expressions. However, according to results of emotion recognition task via facial expressions, there were statistically significant differences between pure ADHD and comorbid ADHD+ODD groups in happy and neutral expressions. ADHD/ODD group tend to attribute more meaning to neutral facial expressions. Additionally there was statistically significant difference between control group and ADHD group according to recognition of angry expressions. There were statistically significant differences between the groups according to recognition of sad expressions in all clear, blurred, and eye photographs.Conclusions: Difficulties in recognizing emotional facial expressions were observed in children with ADHD. A statistically significant association was established between presence of ADHD and impaired recognition of facial emotion expressions independent from the scores of the disruptive behaviour rating scale. Comorbid ODD was not associated with recognition of emotional facial expressions including angry expressions. Recognition of angry expressions was not found as a predictor for disruptive behaviour disorders.

This thesis proposes a novel unified boosting framework. We apply this framework to the several face processing tasks, face detection, facial feature localisation, and pose classification, and use the same boosting algorithm and the same pool of features (local binary features). This is in contrast with the standard approaches that make use of a variety of features and models, for example AdaBoost, cascades of boosted classifiers and Active Appearance Models. The unified boosting framework covers multivariate classification and regression problems and it is achieved by interpreting boosting as optimization in the functional space of the weak learners. Thus a wide range of smooth loss functions can be optimized with the same algorithm. There are two general optimization strategies we propose that extend recent works on TaylorBoost and Variational AdaBoost. The first proposition is an empirical expectation formulation that minimizes the average loss and the second is a variational formulation that includes an additional penalty for large variations between predictions. These two boosting formulations are used to train real-time models using local binary features. This is achieved using look-up-tables as weak learners and multi-block Local Binary Patterns as features. The resulting boosting algorithms are simple, efficient and easily scalable with the available resources. Furthermore, we introduce a novel coarse-to-fine feature selection method to handle high resolution models and a bootstrapping algorithm to sample representative training data from very large pools of data. The proposed approach is evaluated for several face processing tasks. These tasks include frontal face detection (binary classification), facial feature localization (multivariate regression) and pose estimation (multivariate classification). Several studies are performed to assess different optimization algorithms, bootstrapping parametrizations and feature sharing methods (for the multivariate case). The results show good performance for all of these tasks. In addition to this, two other contributions are presented. First, we propose a context-based model for removing the false alarms generated by a given generic face detector. Second, we propose a new face detector that predicts the Jaccard distance between the current location and the ground truth. This allows us to formulate the face detection problem as a regression task.

This paper proposes a novel natural facial expression recognition method that recognizes a sequence of dynamic facial expression images using the differential active appearance model (AAM) and k-NNS as follows. First, we use the differential-AAM features (DAFs) that are computed from the difference of the AAM parameters between an input face image and a reference face image. Second, we perform the manifold learning. Third, we recognize the facial expression of the input face image in the embedded feature space using sequence based k-NN, k-NNS. Since we use DAFs, we also propose an effective way of finding the neutral facial expression as kernel density approximation. Experimental results show that (1) the DAFs improves the facial expression recognition performance than the conventional AAM features by 20% and (2) the sequence-based k-nearest neighbors classifier provides a 95% of facial expression recognition performance on the facial expression database (FED06).

Face recognition is a biometric technology used to identify individuals based on facial feature information. Previous studies of face recognition used features including the eye, mouth and nose; however, there have been few studies on the effects of using other facial components, such as the eyebrows and chin, on recognition performance. We measured the recognition accuracy affected by these facial components, and compared the differences between computer-based and human-based facial recognition methods. This research is novel in the following four ways compared to previous works. First, we measured the effect of components such as the eyebrows and chin. And the accuracy of computer-based face recognition was compared to human-based face recognition according to facial components. Second, for computer-based recognition, facial components were automatically detected using the Adaboost algorithm and active appearance model (AAM), and user authentication was achieved with the face recognition algorithm based on principal component analysis (PCA). Third, we experimentally proved that the number of facial features (when including eyebrows, eye, nose, mouth, and chin) had a greater impact on the accuracy of human-based face recognition, but consistent inclusion of some feature such as chin area had more influence on the accuracy of computer-based face recognition because a computer uses the pixel values of facial images in classifying faces. Fourth, we experimentally proved that the eyebrow feature enhanced the accuracy of computer-based face recognition. However, the problem of occlusion by hair should be solved in order to use the eyebrow feature for face recognition.

Two incompatible pictures compete for perceptual dominance when they are presented to one eye each. This so-called binocular rivalry results in an alternation of dominant and suppressed percepts. In accordance with current theories of emotion processing, the authors' previous research has suggested that emotionally arousing pictures predominate in this perceptual process. Three experiments were run with pictures of emotional facial expressions that are known to induce emotions while being well controlled in terms of physical characteristics. In Experiment 1, photographs of emotional and neutral facial expressions were presented of the same actor to minimize physical differences. In Experiment 2, schematic emotional expressions were presented to further eliminate low-level differences. In Experiment 3, a probe-detection task was conducted to control for possible response-biases. Together, these data clearly demonstrate that emotional facial expressions predominate over neutral expressions; they are more often the first percept and they are perceived for longer durations. This is not caused by physical stimulus properties or by response-biases. This novel approach supports that emotionally significant visual stimuli are preferentially perceived.

Psychiatric social work intervention program for “high expressed emotion” among the caregivers of persons with schizophrenia in India – a feasibility study

Because of high dimensionality, correlation among covariates, and noise contained in data, dimension reduction (DR) techniques are often employed to the application of machine learning algorithms. Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), and their kernel variants (KPCA, KLDA) are among the most popular DR methods. Recently, Supervised Kernel Principal Component Analysis (SKPCA) has been shown as another successful alternative. In this paper, brief reviews of these popular techniques are presented first. We then conduct a comparative performance study based on three simulated datasets, after which the performance of the techniques are evaluated through application to a pattern recognition problem in face image analysis. The gender classification problem is considered on MORPH-II and FG-NET, two popular longitudinal face aging databases. Several feature extraction methods are used, including biologically-inspired features (BIF), local binary patterns (LBP), histogram of oriented gradients (HOG), and the Active Appearance Model (AAM). After applications of DR methods, a linear support vector machine (SVM) is deployed with gender classification accuracy rates exceeding 95% on MORPH-II, competitive with benchmark results. A parallel computational approach is also proposed, attaining faster processing speeds and similar recognition rates on MORPH-II. Our computational approach can be applied to practical gender classification systems and generalized to other face analysis tasks, such as race classification and age prediction.

Objective To explore facial emotion cognition character and the correlation to frontal lobe white matter fiber tracts in first-episode of schizophrenia patients in comparison with healthy control. Method The diffusion tensor imaging data were obtained from 56 patients with untreated first-episode schizophrenia and 30 healthy controls subjects. Fiber tracking was performed in the whole cerebral cortex of brain. The parameter of impairment white matter fiber tracts were obtained,such as fractional anisotropy, mean diffusivity, radial diffusivity, axial diffusivity. Computer-based facial emotion cognition test was used to evaluate difference of facial emotion identification. The correlation between ability of identifying ambiguous emotion and the parameter of impairment white matter fiber tracts, and the correlation between ability of identifying ambiguous emotion and PANSS scale were explored by linear correlation analysis. Result (1) Between patients and healthy control, there was no significant difference in the proportion of recognition as happy emotion to mixture of 50% happy emotion plus 50% angry emotion. In patients group, the number of regarding 100% happy emotion plus 0 angry emotion (9.96±2.42) and regarding 75% happy emotion plus 25% angry emotion (9.91±2.33) both as happy emotion were significantly lower than the healthy control (11.40±1.48, 11.30±1.44; t=-3.332, -3.325; all P<0.01).The number of regarding 25% happy emotion and 75% angry emotion as happy emotion(2.24±2.57), and the number of regarding 0 happy emotion and 100% angry emotion as happy emotion(1.78±2.66) were higher than healthy control(0.83±2.21, 0.70±2.02; t=2.537, 2.056; P<0.05 or P<0.01). The total response time for all images on screen was significantly longer than healthy control(t=6.441, P<0.01).(2)There are five relative serious fiber tracts impaired. Patients with schizophrenia were demonstrated lower activation in right dorsolateral frontal gyru, right inferior frontal operculum,right medial frontal gyrus, left precentral gyrus, left olfactory cortex (t=-2.993 0-2.830 0, P<0.01 or P<0.05), in comparison with healthy controls, except FA value of right inferior frontal operculum and AD value of left precentral gyrus were found higher in patients group. (3) There was no correlation between identifying facial emotion and PANSS. (4)The correlation between identifying ambiguous emotion and the parameter of impairment or white matter fiber tracts was explored by linear correlation analysis, which found FA value of left precentral gyrus positively correlated to the number of identifying happy emotion from 25% happy emotion plus 75% angry emotion (r=0.362, P=0.007), and from 0 happy emotion plus 100% angry emotion (r=0.476, P=0.000). Conclusion This study suggests that schizophrenia patients could not identify facial emotion accurately and their white matter fiber tracts in the frontal lobe are damaged significantly. The impairment of facial emotion recognition is correlated with the abnormality of white matter fiber tracts. Key words: Schizophrenia; Cognition; Frontal lobe; White matter fiber tracts

Over the course of the last decade, infrared (IR) and particularly thermal IR imaging based face recognition has emerged as a promising complement to conventional, visible spectrum based approaches which continue to struggle when applied in the real world. While inherently insensitive to visible spectrum illumination changes, IR images introduce specific challenges of their own, most notably sensitivity to factors which affect facial heat emission patterns, e.g. emotional state, ambient temperature, and alcohol intake. In addition, facial expression and pose changes are more difficult to correct in IR images because they are less rich in high frequency detail which is an important cue for fitting any deformable model. In this paper we describe a novel method which addresses these major challenges. Specifically, to normalize for pose and facial expression changes we generate a synthetic frontal image of a face in a canonical, neutral facial expression from an image of the face in an arbitrary pose and facial expression. This is achieved by piecewise affine warping which follows active appearance model (AAM) fitting. This is the first publication which explores the use of an AAM on thermal IR images; we propose a pre-processing step which enhances detail in thermal images, making AAM convergence faster and more accurate. To overcome the problem of thermal IR image sensitivity to the exact pattern of facial temperature emissions we describe a representation based on reliable anatomical features. In contrast to previous approaches, our representation is not binary; rather, our method accounts for the reliability of the extracted features. This makes the proposed representation much more robust both to pose and scale changes. The effectiveness of the proposed approach is demonstrated on the largest public database of thermal IR images of faces on which it achieved 100% identification rate, significantly outperforming previously described methods.

Dynamic evoked facial expressions of emotions in schizophrenia

The effect of gender-role stereotypes means that the leadership effectiveness of female leaders is likely to be underestimated, as their roles are different from those of traditional male leaders. Given the theory of paternalistic leadership in Chinese society and from the perspective of Chinese culture, this study re-examined the relationship between the emotional expression and leadership style of supervisors and gender-role stereotypes and its effect on leadership effectiveness. Data were collected using questionnaires distributed to supervisor-subordinate pairs in various departments of 76 enterprises in Taiwan. The research findings showed that gender differences among supervisors did not have a direct effect on leadership effectiveness. However, with the moderation of gender differences in the emotional expression and leadership style of supervisors, negative emotional expression, authoritarian leadership and benevolent leadership all had a significant effect on leadership effectiveness for both genders. The implication of these findings was that gender does not directly affect leadership effectiveness, but that the display of emotional expression and a leadership style consistent with gender expectations, does have effect on the leadership effectiveness of supervisors; thus, providing evidence of the prevalence of gender-role stereotypes in Chinese societies. Key words: Emotional expression, paternalistic leadership, leadership effectiveness, gender-role stereotypes.

Electrocortical evidence for preferential processing of dynamic pain expressions compared to other emotional expressions

Although recent studies have convincingly demonstrated that emotional expressions can be judged reliably from actor-posed facial displays, there exists little evidence that facial expressions in lifelike settings are similar to actor-posed displays, are reliable across situations designed to elicit the same emotion, or provide sufficient information to mediate consistent emotion judgments by raters. The present study therefore investigated these issues as they related to the emotions of happiness, surprise, and fear. 27 infants between 10 and 12 months of age (when emotion masking is not likely to confound results) were tested in 2 situations designed to elicit hapiness (peek-a-boo game and a collapsing toy), 2 to elicit surprise (a toy-switch and a vanishing-object task), and 2 to elicit fear (the visual cliff and the approach of a stranger. Dependent variables included changes in 28 facial response components taken from previous work using actor poses, as well as judgments of the presence of 6 discrete emotions. In addition, instrumental behaviors were used to verify with other than facial expression responses whether the predicted emotion was elicited. In contrast to previous conclusions on the subject, we found that judges were able to make all facial expression judgments reliably, even in the absence of contextual information. Support was also obtained for at least some degree of specificity of facial component response patterns, especially for happiness and surprise. Emotion judgments by raters were found to be a function of the presence of discrete facial components predicted to be linked to those emotions. Finally, almost all situations elicited blends, rather than discrete emotions.

This study aimed to differentiate hip-hop dance steps based on whether joy was expressed and explained their movement features using three-dimensional motion analysis. Ten male and female participants (26.9 ± 12.8 years old) were instructed to perform an 80-second hip-hop dance, and the step performed after 60 s (new jack swing) was analyzed. Discriminant analysis was conducted, and the differences in the motion feature values were calculated by establishing 34 variables related to time and space. Furthermore, correlations between major joint sites that contributed to differentiation and other variables were calculated. The results revealed the following three points: The discriminant analysis, which was used to predict whether there was emotion, correctly classified cases at a rate of 70% or higher in four stages, which were motion phases. When the differences in motion features by intensity were identified in Phase 2—the phase with the highest correct classification rate—significant differences were found in the mean of the height in the shoulders, the speed of the left acromion, the speed of the capitellum of the left elbow joint, the speed of the center of the left wrist joint, the speed of the lateral condyle of the left femur, the speed of the right acromion, the speed of the capitellum of the right elbow joint, and the flexion angle of the upper torso. Under Strong Emotion, the speed of the left elbow joint and the speed of the center of the left wrist joint had the highest correlation (r = 0.92). Based on these results, we can conclude that it is possible to express joy through hip-hop dance, and, in the case of new jack swing, emotional expression is related mainly to the speed of the upper limbs. Key words: Hip-hop dance, dance step, emotional expression.