Fusing concurrent EEG–fMRI with dynamic causal modeling: Application to effective connectivity during face perception

Event Abstract Back to Event An MEG neuroimaging study on the developmental changes of face processing in pre-school aged children Wei He1*, Jon Brock1 and Blake W. Johnson1 1 Macquarie University, Department of Cognitive Science, Australia Introduction: An important research question in developmental neuroscience concerns how the underlying neural mechanisms of face perception develop in early childhood. Magnetoencephalography (MEG) holds great promise for examining this question, because it measures a well-known face sensitive brain response, termed the M170, from young children in an entirely passive manner. The M170 and its electrical equivalent N170 discriminate faces from other non-face objects based on larger amplitudes to face stimuli around 170 ms from stimulus onsets in adults (Rossion & Jacques, 2011). The current study measured the M170 from pre-schoolers using a unique child custom-sized magnetoencephalography (MEG) system (Johnson et al., 2010). Furthermore, we evaluated the effective connectivity of cortical networks underlying the M170 using dynamic causal modelling (DCM); and compared the connectivity patterns in children to those of adults. Methods: Brain responses were measured from 15 young children (aged 4.46 ± 0.93 years) and 15 adults (aged 27.6 ± 6.46 years) during flashing pictures of faces, cars, as well as their phase-scrambled counterparts (Kuefner et al, 2010). There were 64 channels in the child MEG system and 160 channels in the adult MEG system. Surface waveforms were analysed by averaging the strongest response across occipital and temporal sensors separately on each hemisphere based on the time window determined by visual checking on individual data. DCM analyses assumed that the M170 was mediated by sources in the occipital face area (OFA), fusiform face area (FFA), and superior temporal sulcus (STS). Using Bayesian model selection, we evaluated three models of the interconnections between these regions based on previous studies using DCM (Fairhall & Ishai, 2007; Chen, et al., 2009; David et al., 2006). Results: In the surface waveform analysis, the child M170 was slower than the adult M170, but it already showed an adult function pattern (i.e., faster and stronger to faces than cars). Analyses of DCM models showed that the first and simplest model with only forward intra-hemispheric connections from OFA to FFA and STS had the largest model evidence in adults, while in children, the second model with extra inter-hemispheric connections between OFA and contralateral FFA showed the largest model evidence. Conclusions: Early face sensitive brain responses indexed by the M170 are present and functioning in the adult pattern in children as young as 4 years old. However, the functional neural mechanism underlying the M170 undergoes further development and fine-tuning before it reaches adult capacities. Acknowledgements This work was supported by Australian Research Council Linkage Infrastructure Equipment and Facilities Grant LEO668421, Australian Research Council Linkage Project Grant LP0669471, and the Australian Research Council Centre of Excellence for Cognition and its Disorders (CE110001021), http://www.ccd.edu.au. The authors thank Romina Polermo and Douglas Cheyne for helpful comments during the design of the experiment and Bruno Russion for providing the picture stimuli.

Patterns of neural response in face regions are predicted by low-level image properties

Electrophysiological studies of adults indicate that brain activity is enhanced during viewing of repeated faces, at a latency of about 250 ms after the onset of the face (M250/N250). The present study aimed to determine if this effect was also present in preschool-aged children, whose brain activity was measured in a custom-sized pediatric MEG system. The results showed that, unlike adults, face repetition did not show any significant modulation of M250 amplitude in children; however children’s M250 latencies were significantly faster for repeated than non-repeated faces. Dynamic causal modelling (DCM) of the M250 in both age groups tested the effects of face repetition within the core face network including the occipital face area (OFA), the fusiform face area (FFA), and the superior temporal sulcus (STS). DCM revealed that repetition of identical faces altered both forward and backward connections in children and adults; however the modulations involved inputs to both FFA and OFA in adults but only to OFA in children. These findings suggest that the amplitude-insensitivity of the immature M250 may be due to a weaker connection between the FFA and lower visual areas.

Perception of faces has been shown to engage a domain-specific set of brain regions, including the occipital face area (OFA) and the fusiform face area (FFA). It is commonly held that the OFA is responsible for the detection of faces in the environment, whereas the FFA is responsible for processing the identity of the face. However, an alternative model posits that the FFA is responsible for face detection and subsequently recruits the OFA to analyze the face parts in the service of identification. An essential prediction of the former model is that the OFA is not sensitive to the arrangement of internal face parts. In the current fMRI study, we test the sensitivity of the OFA and FFA to the configuration of face parts. Participants were shown faces in which the internal parts were presented in a typical configuration (two eyes above a nose above a mouth) or in an atypical configuration (the locations of individual parts were shuffled within the face outline). Perception of the atypical faces evoked a significantly larger response than typical faces in the OFA and in a wide swath of the surrounding posterior occipitotemporal cortices. Surprisingly, typical faces did not evoke a significantly larger response than atypical faces anywhere in the brain, including the FFA (although some subthreshold differences were observed). We propose that face processing in the FFA results in inhibitory sculpting of activation in the OFA, which accounts for this region's weaker response to typical than to atypical configurations.

Repetition of identical face stimuli leads to fMRI response attenuation (fMRI adaptation, fMRIa) in the core face-selective occipito-temporal visual cortical network, involving the bilateral fusiform face area (FFA) and the occipital face area (OFA). However, the functional relevance of fMRIa observed in these regions is unclear as of today. Therefore, here we aimed at investigating the relationship between fMRIa and face perception ability by measuring in the same human participants both the repetition-induced reduction of fMRI responses and identity discrimination performance outside the scanner for upright and inverted face stimuli. In the correlation analysis, the behavioral and fMRI results for the inverted faces were used as covariates to control for the individual differences in overall object perception ability and basic visual feature adaptation processes, respectively. The results revealed a significant positive correlation between the participants' identity discrimination performance and the strength of fMRIa in the core face processing network, but not in the extrastriate body area (EBA). Furthermore, we found a strong correlation of the fMRIa between OFA and FFA and also between OFA and EBA, but not between FFA and EBA. These findings suggest that there is a face-selective component of the repetition-induced reduction of fMRI responses within the core face processing network, which reflects functionally relevant adaptation processes involved in face identity perception.

Face recognition and memory entail not only encoding the perceptual input of a face upon its presence but also retrieving a relatively permanent representation in spite of variation in illumination, pose, an/or expression. For more than two decades, a network of face-selective regions has been identified as the core system of face processing, including occipital face area (OFA), fusiform face area (FFA), and posterior region of superior temporal sulcus (pSTS). However, recent studies have proposed that ventral route of face processing and memory ends at the anterior temporal lobes (i.e., vATLs), which may play an important role bridging face perception and face memory. Here we examined whether neural activity in vATLs can effectively predict performance on a face memory test that requires recognition circumventing variations in pose and lighting. To that end, we first identified during the functional scan the core face network by asking participants to perform a one-back task, while viewing either static images or dynamic videos. Compared to static localizers, dynamic localizers were far more effective identifying regions-of-interest (ROIs) in the core face processing system. We then determined for each ROI (OFA, FFA, pSTS, and vATL), the cluster size associated with maximum face selectivity. Participants were called back with various delays to perform a variety of face processing tasks, including the Taiwanese Face Memory Test (TFMT), which was constructed largely following Cambridge Face Memory Task (CFMT) and used images drawn from a recently established Taiwanese face database. Like CFMT, TFMT was administered in three consecutive stages with increasing reliance on robust face representations. Correlation analyses revealed that participants with greater neural adaptation in the right vATLs demonstrated better recognition and memory performance on TFMT, suggesting that individual differences in constructing invariant and robust neural representation of faces can predict behavioural performance on face recognition and memory. Meeting abstract presented at VSS 2016

Macaque neurophysiology found image-invariant representations of face identity in a face-selective patch in anterior temporal cortex. A face-selective area in human anterior temporal lobe (fATL) has been reported, but has not been reliably identified, and its function and relationship with posterior face areas is poorly understood. Here, we used fMRI adaptation and neuropsychology to ask whether fATL contains image-invariant representations of face identity, and if so, whether these representations require normal functioning of fusiform face area (FFA) and occipital face area (OFA). We first used a dynamic localizer to demonstrate that 14 of 16 normal subjects exhibit a highly selective right fATL. Next, we found evidence that this area subserves image-invariant representation of identity: Right fATL showed repetition suppression to the same identity across different images, while other areas did not. Finally, to examine fATL's relationship with posterior areas, we used the same procedures with Galen, an acquired prosopagnosic who lost right FFA and OFA. Despite the absence of posterior face areas, Galen's right fATL preserved its face selectivity and showed repetition suppression comparable to that in controls. Our findings suggest that right fATL contains image-invariant face representations that can persist despite the absence of right FFA and OFA, but these representations are not sufficient for normal face recognition.

SummaryHumans show a remarkable ability to discriminate others' gaze direction, even though a given direction can be conveyed by many physically dissimilar configurations of different eye positions and head views. For example, eye contact can be signaled by a rightward glance in a left-turned head or by direct gaze in a front-facing head. Such acute gaze discrimination implies considerable perceptual invariance. Previous human research found that superior temporal sulcus (STS) responds preferentially to gaze shifts [1], but the underlying representation that supports such general responsiveness remains poorly understood. Using multivariate pattern analysis (MVPA) of human functional magnetic resonance imaging (fMRI) data, we tested whether STS contains a higher-order, head view-invariant code for gaze direction. The results revealed a finely graded gaze direction code in right anterior STS that was invariant to head view and physical image features. Further analyses revealed similar gaze effects in left anterior STS and precuneus. Our results suggest that anterior STS codes the direction of another's attention regardless of how this information is conveyed and demonstrate how high-level face areas carry out fine-grained, perceptually relevant discrimination through invariance to other face features.

Face specific broadband electrocorticographic spectral power change in the rhinal cortex

Effective connectivity analysis demonstrates involvement of premotor cortex during speech perception

Understanding how the human visual system recognizes objects is one of the key challenges in neuroscience. Inspired by a large body of physiological evidence, a general class of recognition models has emerged, which is based on a hierarchical organization of visual processing, with succeeding stages being sensitive to image features of increasing complexity. However, these models appear to be incompatible with some well-known psychophysical results. Prominent among these are experiments investigating recognition impairments caused by vertical inversion of images, especially those of faces. It has been reported that faces that differ 'featurally' are much easier to distinguish when inverted than those that differ 'configurally'; a finding that is difficult to reconcile with the physiological models. Here, we show that after controlling for subjects' expectations, there is no difference between 'featurally' and 'configurally' transformed faces in terms of inversion effect. This result reinforces the plausibility of simple hierarchical models of object representation and recognition in the cortex.

Effective connectivity (EC) refers to directional or causal influences between interacting neuronal populations or brain regions and can be estimated from functional magnetic resonance imaging (fMRI) data via dynamic causal modeling (DCM). In contrast to functional connectivity, the impact of data processing varieties on DCM estimates of task‐evoked EC has hardly ever been addressed. We therefore investigated how task‐evoked EC is affected by choices made for data processing. In particular, we considered the impact of global signal regression (GSR), block/event‐related design of the general linear model (GLM) used for the first‐level task‐evoked fMRI analysis, type of activation contrast, and significance thresholding approach. Using DCM, we estimated individual and group‐averaged task‐evoked EC within a brain network related to spatial conflict processing for all the parameters considered and compared the differences in task‐evoked EC between any two data processing conditions via between‐group parametric empirical Bayes (PEB) analysis and Bayesian data comparison (BDC). We observed strongly varying patterns of the group‐averaged EC depending on the data processing choices. In particular, task‐evoked EC and parameter certainty were strongly impacted by GLM design and type of activation contrast as revealed by PEB and BDC, respectively, whereas they were little affected by GSR and the type of significance thresholding. The event‐related GLM design appears to be more sensitive to task‐evoked modulations of EC, but provides model parameters with lower certainty than the block‐based design, while the latter is more sensitive to the type of activation contrast than is the event‐related design. Our results demonstrate that applying different reasonable data processing choices can substantially alter task‐evoked EC as estimated by DCM. Such choices should be made with care and, whenever possible, varied across parallel analyses to evaluate their impact and identify potential convergence for robust outcomes.

Faces are among the most important visual stimuli we perceive, informing us not only about a person's identity, but also about their mood, sex, age and direction of gaze. The ability to extract this information within a fraction of a second of viewing a face is important for normal social interactions and has probably played a critical role in the survival of our primate ancestors. Considerable evidence from behavioural, neuropsychological and neurophysiological investigations supports the hypothesis that humans have specialized cognitive and neural mechanisms dedicated to the perception of faces (the face-specificity hypothesis). Here, we review the literature on a region of the human brain that appears to play a key role in face perception, known as the fusiform face area (FFA). Section 1 outlines the theoretical background for much of this work. The face-specificity hypothesis falls squarely on one side of a longstanding debate in the fields of cognitive science and cognitive neuroscience concerning the extent to which the mind/brain is composed of: (i) special-purpose ('domain-specific') mechanisms, each dedicated to processing a specific kind of information (e.g. faces, according to the face-specificity hypothesis), versus (ii) general-purpose ('domain-general') mechanisms, each capable of operating on any kind of information. Face perception has long served both as one of the prime candidates of a domain-specific process and as a key target for attack by proponents of domain-general theories of brain and mind. Section 2 briefly reviews the prior literature on face perception from behaviour and neurophysiology. This work supports the face-specificity hypothesis and argues against its domain-general alternatives (the individuation hypothesis, the expertise hypothesis and others). Section 3 outlines the more recent evidence on this debate from brain imaging, focusing particularly on the FFA. We review the evidence that the FFA is selectively engaged in face perception, by addressing (and rebutting) five of the most widely discussed alternatives to this hypothesis. In section 4, we consider recent findings that are beginning to provide clues into the computations conducted in the FFA and the nature of the representations the FFA extracts from faces. We argue that the FFA is engaged both in detecting faces and in extracting the necessary perceptual information to recognize them, and that the properties of the FFA mirror previously identified behavioural signatures of face-specific processing (e.g. the face-inversion effect). Section 5 asks how the computations and representations in the FFA differ from those occurring in other nearby regions of cortex that respond strongly to faces and objects. The evidence indicates clear functional dissociations between these regions, demonstrating that the FFA shows not only functional specificity but also area specificity. We end by speculating in section 6 on some of the broader questions raised by current research on the FFA, including the developmental origins of this region and the question of whether faces are unique versus whether similarly specialized mechanisms also exist for other domains of high-level perception and cognition.

Our understanding of facial emotion perception has been dominated by two seemingly opposing theories: the categorical and dimensional theories. However, we have recently demonstrated that hybrid processing involving both categorical and dimensional perception can be induced in an implicit manner (Fujimura etal., 2012). The underlying neural mechanisms of this hybrid processing remain unknown. In this study, we tested the hypothesis that separate neural loci might intrinsically encode categorical and dimensional processing functions that serve as a basis for hybrid processing. We used functional magnetic resonance imaging to measure neural correlates while subjects passively viewed emotional faces and performed tasks that were unrelated to facial emotion processing. Activity in the right fusiform face area (FFA) increased in response to psychologically obvious emotions and decreased in response to ambiguous expressions, demonstrating the role of the FFA in categorical processing. The amygdala, insula and medial prefrontal cortex exhibited evidence of dimensional (linear) processing that correlated with physical changes in the emotional face stimuli. The occipital face area and superior temporal sulcus did not respond to these changes in the presented stimuli. Our results indicated that distinct neural loci process the physical and psychological aspects of facial emotion perception in a region-specific and implicit manner.

We tested functional activation for faces in patient D.F., who following acquired brain damage has a profound deficit in object recognition based on form (visual form agnosia) and also prosopagnosia that is undocumented to date. Functional imaging demonstrated that like our control observers, D.F. shows significantly more activation when passively viewing face compared to scene images in an area that is consistent with the fusiform face area (FFA) ( p < 0.01). Control observers also show occipital face area (OFA) activation; however, whereas D.F.'s lesions appear to overlap the OFA bilaterally. We asked, given that D.F. shows FFA activation for faces, to what extent is she able to recognize faces? D.F. demonstrated a severe impairment in higher level face processing—she could not recognize face identity, gender or emotional expression. In contrast, she performed relatively normally on many face categorization tasks. D.F. can differentiate faces from non-faces given sufficient texture information and processing time, and she can do this is independent of color and illumination information. D.F. can use configural information for categorizing faces when they are presented in an upright but not a sideways orientation and given that she also cannot discriminate half-faces she may rely on a spatially symmetric feature arrangement. Faces appear to be a unique category, which she can classify even when she has no advance knowledge that she will be shown face images. Together, these imaging and behavioral data support the importance of the integrity of a complex network of regions for face identification, including more than just the FFA—in particular the OFA, a region believed to be associated with low-level processing.