Are expression aftereffects fully explained by tilt adaptation?

How do we visually encode facial expressions? Is this done by viewpoint-dependent mechanisms representing facial expressions as two-dimensional templates or do we build more complex viewpoint independent three-dimensional representations? Recent facial adaptation techniques offer a powerful way to address these questions. Prolonged viewing of a stimulus (adaptation) changes the perception of subsequently viewed stimuli (an after-effect). Adaptation to a particular attribute is believed to target those neural mechanisms encoding that attribute. We gathered images of facial expressions taken simultaneously from five different viewpoints evenly spread from the three-quarter leftward to the three-quarter rightward facing view. We measured the strength of expression after-effects as a function of the difference between adaptation and test viewpoints. Our data show that, although there is a decrease in after-effect over test viewpoint, there remains a substantial after-effect when adapt and test are at differing three-quarter views. We take these results to indicate that neural systems encoding facial expressions contain a mixture of viewpoint-dependent and viewpoint-independent elements. This accords with evidence from single cell recording studies in macaque and is consonant with a view in which viewpoint-independent expression encoding arises from a combination of view-dependent expression-sensitive responses.

Le coaching n’est pas de la psychothérapie !

Perhaps no other art form relies upon the expressive and communicative potential of the human face quite as much as narrative cinema. It comes as no surprise then that the face - especially as it appears in the close-up - has been a subject of fascination for film theory for over a century now. The past thirty years have seen the face once again return to the fore in theorising about film. In particular, the human face has occupied a special place of interest for the field of cognitive film theory. Cognitive film theory has examined various aspects of the face in recent years: how viewers come to recognise emotions from facial expressions, how cinema can aesthetically 'sculpt' ordinary forms of human expression, and how cinema may elicit empathetic responses through representations of the face are but a few of the topics that have been addressed. Despite the long history of scholarship on the face in film and the recent work within cognitive film theory, there remain numerous unexplored avenues of research. For example, one of the biggest controversies in the scientific study of facial expression is the matter of universality and cultural difference. Are the faces people make the same the world over? Are people equally adept at recognising facial expressions of individuals from cultures beyond their own? And from where do cultural differences in facial expression arise? Although these questions pertain to scientific research on facial expression in everyday life, I argue that the answers to these problems are nonetheless highly significant for our understanding of cinema. This thesis thus responds to two central questions that have hitherto not been addressed in detail in film theory: how do cultural differences shape the representation of faces in narrative cinema? And do film viewers across different cultures recognise and understand faces and facial expressions in substantially different ways? To respond to these questions, this thesis makes the case that we should adopt a 'cognitive cultural' approach. Carl Plantinga has recently proposed that such an approach can account for the mixture of dispositions at work in the viewer's experience of faces in film, since the cognitive cultural approach is explicitly interested in both the universal and the culturally specific. As Lisa Zunshine puts it, the goal of the cognitive cultural project is to make sense of the ever-changing relationship between two highly complex and historically situated systems: cultural artefacts and the human mind. This thesis takes Plantinga's proposal further and aims to show the benefits of adopting a cognitive cultural approach to faces in film. To this end, the first half of the thesis examines the dominant views of facial expression within psychology, explores the implications of cultural differences for understanding film viewership, and presents a general account of how faces and facial expression are represented in film. The second half builds upon this theoretical groundwork and works through three different case studies that demonstrate what is gained from a cognitive cultural approach to faces in film. Ultimately, this thesis advocates for a middle way through the unproductive dichotomies between nature and nurture, universality and cultural difference, and (perhaps above all) cognitivist studies and cultural studies.

The expression of human emotion is integral to social interaction, and in virtual reality it is increasingly common to develop virtual avatars that attempt to convey emotions by mimicking these visual and aural cues, i.e. the facial and vocal expressions. However, errors in (or the absence of) facial tracking can result in the rendering of incorrect facial expressions on these virtual avatars. For example, a virtual avatar may speak with a happy or unhappy vocal inflection while their facial expression remains otherwise neutral. In circumstances where there is conflict between the avatar's facial and vocal expressions, it is possible that users will incorrectly interpret the avatar's emotion, which may have unintended consequences in terms of social influence or in terms of the outcome of the interaction. In this paper, we present a human-subjects study (N = 22) aimed at understanding the impact of conflicting facial and vocal emotional expressions. Specifically we explored three levels of emotional valence (unhappy, neutral, and happy) expressed in both visual (facial) and aural (vocal) forms. We also investigate three levels of head scales (down-scaled, accurate, and up-scaled) to evaluate whether head scale affects user interpretation of the conveyed emotion. We find significant effects of different multimodal expressions on happiness and trust perception, while no significant effect was observed for head scales. Evidence from our results suggest that facial expressions have a stronger impact than vocal expressions. Additionally, as the difference between the two expressions increase, the less predictable the multimodal expression becomes. For example, for the happy-looking and happy-sounding multimodal expression, we expect and see high happiness rating and high trust, however if one of the two expressions change, this mismatch makes the expression less predictable. We discuss the relationships, implications, and guidelines for social applications that aim to leverage multimodal social cues.

Despite the importance of perceiving and recognizing facial expressions in everyday life, there is no comprehensive test battery for the multivariate assessment of these abilities. As a first step toward such a compilation, we present 16 tasks that measure the perception and recognition of facial emotion expressions, and data illustrating each task's difficulty and reliability. The scoring of these tasks focuses on either the speed or accuracy of performance. A sample of 269 healthy young adults completed all tasks. In general, accuracy and reaction time measures for emotion-general scores showed acceptable and high estimates of internal consistency and factor reliability. Emotion-specific scores yielded lower reliabilities, yet high enough to encourage further studies with such measures. Analyses of task difficulty revealed that all tasks are suitable for measuring emotion perception and emotion recognition related abilities in normal populations.

Background: Individuals with child maltreatment (CM) experiences show alterations in emotion recognition (ER). However, previous research has mainly focused on populations with specific mental disorders, which makes it unclear whether alterations in the recognition of facial expressions are related to CM, to the presence of mental disorders or to the combination of CM and mental disorders, and on ER of emotional, rather than neutral facial expressions. Moreover, commonly, recognition of static stimulus material was researched. Objective: We assessed recognition of dynamic (closer to real life) negative, positive and neutral facial expressions in individuals characterised by CM, rather than a specific mental disorder. Moreover, we assessed whether they show a negativity bias for neutral facial expressions and whether the presence of one or more mental disorders affects recognition. Methods: Ninety-eight adults with CM experiences (CM+) and 60 non-maltreated (CM−) adult controls watched 200 non-manipulated coloured video sequences, showing 20 neutral and 180 emotional facial expressions, and indicated whether they interpreted each expression as neutral or as one of eight emotions. Results: The CM+ showed significantly lower scores in the recognition of positive, negative and neutral facial expressions than the CM− group (p < .050). Furthermore, the CM+ group showed a negativity bias for neutral facial expressions (p < .001). When accounting for mental disorders, significant effects stayed consistent, except for the recognition of positive facial expressions: individuals from the CM+ group with but not without mental disorder scored lower than controls without mental disorder. Conclusions: CM might have long-lasting influences on the ER abilities of those affected. Future research should explore possible effects of ER alterations on everyday life, including implications of the negativity bias for neutral facial expressions on emotional wellbeing and relationship satisfaction, providing a basis for interventions that improve social functioning.

Cortical responses to dynamic emotional facial expressions generalize across stimuli, and are sensitive to task-relevance, in adults with and without Autism

Converging evidence suggests that infants can extract and integrate emotional content from multiple sources (e.g., faces, body postures, and voices). Yet this evidence is mostly based on static representations of emotions, such as photographs, whereas in everyday life, infants are primarily exposed to dynamic input, particularly others' actions. This study investigates whether infants can link emotional information conveyed in action kinematics and facial expressions. To address this issue, we used an ERP priming paradigm in which 12-month-olds were presented with video primes of actions performed with happy or angry kinematics, followed by target images of faces displaying happy or angry facial expressions. Results revealed a P400 congruency effect over the right hemisphere. Specifically, happy faces elicited a larger P400 than angry faces when they followed an incongruent emotional action. Moreover, the P400 was larger for incongruent than for congruent happy facial expressions. Results suggest that bodily kinematics provide infants with crucial contextual and emotional cues that bias their perception of facial expressions from early in life.

Visual adaptation is a powerful tool to probe the short-term plasticity of the visual system. Adapting to local features such as the oriented lines can distort our judgment of subsequently presented lines, the tilt aftereffect. The tilt aftereffect is believed to be processed at the low-level of the visual cortex, such as V1. Adaptation to faces, on the other hand, can produce significant aftereffects in high-level traits such as identity, expression, and ethnicity. However, whether face adaptation necessitate awareness of face features is debatable. In the current study, we investigated whether facial expression aftereffects (FEAE) can be generated by partially visible faces. We first generated partially visible faces using the bubbles technique, in which the face was seen through randomly positioned circular apertures, and selected the bubbled faces for which the subjects were unable to identify happy or sad expressions. When the subjects adapted to static displays of these partial faces, no significant FEAE was found. However, when the subjects adapted to a dynamic video display of a series of different partial faces, a significant FEAE was observed. In both conditions, subjects could not identify facial expression in the individual adapting faces. These results suggest that our visual system is able to integrate unrecognizable partial faces over a short period of time and that the integrated percept affects our judgment on subsequently presented faces. We conclude that FEAE can be generated by partial face with little facial expression cues, implying that our cognitive system fills-in the missing parts during adaptation, or the subcortical structures are activated by the bubbled faces without conscious recognition of emotion during adaptation.

Task-dependent enhancement of facial expression and identity representations in human cortex

Learning realistic facial expressions from web images

Individuals with autistic traits display impaired social interaction and communication in everyday life, but the underlying cognitive neural mechanisms remain very unclear and still remain controversial. The mind-blindness hypothesis suggests that social difficulties in individuals with autistic traits are caused by empathy impairment in individuals; however, the intense world theory suggests that these social difficulties are caused by sensory hyper-reactivity and sensory overload, rather than empathy impairment. To further test these two theories, this study investigated event-related potentials (ERPs) to explore the cognitive neural processing of repetitive expressions in individuals with autistic traits. This study employed the Mandarin version of the autism-spectrum quotient (AQ) to assess autistic traits in 2,502 healthy adults. Two subset groups were used, e.g., the participants of a high-AQ group were randomly selected among the 10% of individuals with the highest AQ scores; similarly, the participants in the low-AQ group were randomly selected from the 10% of participants with the lowest AQ scores. In an experiment, three different facial expressions (positive, neutral, or negative) of the same person were presented successively and pseudo-randomly in each trial. Participants needed to define the expression of the face that was presented last. The results showed that compared with the low-AQ group, the high-AQ group exhibited higher P1 amplitudes induced by the second and third presented expressions, as well as higher P3 amplitudes induced by the third presented negative expressions. This indicates that individuals with autistic traits may experience overly strong perception, attention, and cognitive evaluation to repetitive expressions, particularly negative expressions. This result supports the intense world theory more strongly than the mind-blindness hypothesis.

To date, several studies have found the effect of facial expressions on trust decision, using the event-related potentials (ERPs). However, little is known about the neural mechanism underlying the modulation effect of facial expressions on making choices and subsequent outcome evaluation. In the present study, using an ERP technique, we investigated how the neural process of making choices and subsequent outcome evaluation were influenced by others' facial expressions for the first time. Specifically, participants played a modified version of the Trust Game, in which they watched a photo of the trustee before making choices. Critically, trustees' faces differed regarding emotional types (i.e., happy, neutral, or angry) and gender (i.e., female or male). Behaviorally, an interaction between expressions and gender was observed on investment rates. On the neural level, the N2 and P3 amplitudes were modulated by facial expressions in the making-choice stage. Additionally, the feedback-related P3 was also modulated by facial expressions. The present study proved the effect of facial expressions on making choices and subsequent outcome evaluation.

Processing facial expressions is an essential component of social interaction, especially for preverbal infants. In human adults and monkeys, this process involves the motor system, with a neural matching mechanism believed to couple self- and other-generated facial gestures. Here, we used electroencephalography to demonstrate recruitment of the human motor system during observation and execution of facial expressions in nine-month-old infants, implicating this system in facial expression processing from a very young age. Notably, examination of early video-recorded mother-infant interactions supported the common, but as yet untested, hypothesis that maternal mirroring of infant facial gestures is central to the development of a neural matching mechanism for these gestures. Specifically, the extent to which mothers mirrored infant facial expressions at two months postpartum predicted infant motor system activity during observation of the same expressions at nine months. This suggests that maternal mirroring strengthens mappings between visual and motor representations of facial gestures, which increases infant neural sensitivity to particularly relevant cues in the early social environment.

Early processing of the six basic facial emotional expressions