Dual Neural Routing of Visual Facilitation in Speech Processing

Visual information about lip and facial movements plays a role in audiovisual (AV) speech perception. Although this has been widely confirmed, previous behavioural studies have shown interlanguage differences, that is, native Japanese speakers do not integrate auditory and visual speech as closely as native English speakers. To elucidate the neural basis of such interlanguage differences, 22 native English speakers and 24 native Japanese speakers were examined in behavioural or functional Magnetic Resonance Imaging (fMRI) experiments while mono-syllabic speech was presented under AV, auditory-only, or visual-only conditions for speech identification. Behavioural results indicated that the English speakers identified visual speech more quickly than the Japanese speakers, and that the temporal facilitation effect of congruent visual speech was significant in the English speakers but not in the Japanese speakers. Using fMRI data, we examined the functional connectivity among brain regions important for auditory-visual interplay. The results indicated that the English speakers had significantly stronger connectivity between the visual motion area MT and the Heschl’s gyrus compared with the Japanese speakers, which may subserve lower-level visual influences on speech perception in English speakers in a multisensory environment. These results suggested that linguistic experience strongly affects neural connectivity involved in AV speech integration.

The paper discusses the changes in relations of visual and auditory inputs between the hemispheres in a child with hyperactive syndrome and its effects which may lead to better attention engagement in auditory and visual information analysis. The method included the use of cartographic EEG and clinical procedure in a 10-year-old boy with hyperactive syndrome and attention deficit disorder, who has theta dysfunction manifested in standard EEG. Cartographic EEG patterns was performed on NihonKohden Corporation, EEG – 1200K Neurofax apparatus in longitudinal bipolar electrode assembly schedule by utilizing10/20 International electrode positioning. Impedance was maintained below 5 kΩ, with not more than 1 kΩ differences between the electrodes. Lower filter was set at 0.53 Hz and higher filter at 35 Hz. Recording was performed in a quiet period and during stimulation procedures that include speech and language basis.Standard EEG and Neurofeedback (NFB) treatment indicated
\nhigher theta load, alpha 2 and beta 1 activity measured in the
\ncartographic EEG which was done after the relative failure of NFB
\ntreatment. After this, the NFB treatment was applied which lasted for
\nsix months, in a way that when the boy was reading, the visual input
\nwas enhanced to the left hemisphere and auditory input was reduced
\nto the right hemisphere. Repeated EEG mapping analysis showed that
\nthere was a significant improvement, both in EEG findings as well as
\nin attention, behavioural and learning disorders. The paper discusses
\nsome aspects of learning, attention and behaviour in relation to
\nchanges in the standard EEG, especially in cartographic EEG and
\nNFB findings.

SummaryHumans show a remarkable ability to discriminate others' gaze direction, even though a given direction can be conveyed by many physically dissimilar configurations of different eye positions and head views. For example, eye contact can be signaled by a rightward glance in a left-turned head or by direct gaze in a front-facing head. Such acute gaze discrimination implies considerable perceptual invariance. Previous human research found that superior temporal sulcus (STS) responds preferentially to gaze shifts [1], but the underlying representation that supports such general responsiveness remains poorly understood. Using multivariate pattern analysis (MVPA) of human functional magnetic resonance imaging (fMRI) data, we tested whether STS contains a higher-order, head view-invariant code for gaze direction. The results revealed a finely graded gaze direction code in right anterior STS that was invariant to head view and physical image features. Further analyses revealed similar gaze effects in left anterior STS and precuneus. Our results suggest that anterior STS codes the direction of another's attention regardless of how this information is conveyed and demonstrate how high-level face areas carry out fine-grained, perceptually relevant discrimination through invariance to other face features.

Empowering Reentrant Projections from V5 to V1 Boosts Sensitivity to Motion

Nausea is associated with significant morbidity, and there is a wide range in the propensity of individuals to experience nausea. The neural basis of the heterogeneity in nausea susceptibility is poorly understood. Our previous functional magnetic resonance imaging (fMRI) study in healthy adults showed that a visual motion stimulus caused activation in the right MT+/V5 area, and that increased sensation of nausea due to this stimulus was associated with increased activation in the right anterior insula. For the current study, we hypothesized that individual differences in visual motion-induced nausea are due to microstructural differences in the inferior fronto-occipital fasciculus (IFOF), the white matter tract connecting the right visual motion processing area (MT+/V5) and right anterior insula. To test this hypothesis, we acquired diffusion tensor imaging data from 30 healthy adults who were subsequently dichotomized into high and low nausea susceptibility groups based on the Motion Sickness Susceptibility Scale. We quantified diffusion along the IFOF for each subject based on axial diffusivity (AD); radial diffusivity (RD), mean diffusivity (MD) and fractional anisotropy (FA), and evaluated between-group differences in these diffusion metrics. Subjects with high susceptibility to nausea rated significantly (P < 0.001) higher nausea intensity to visual motion stimuli and had significantly (P < 0.05) lower AD and MD along the right IFOF compared to subjects with low susceptibility to nausea. This result suggests that differences in white matter microstructure within tracts connecting visual motion and nausea-processing brain areas may contribute to nausea susceptibility or may have resulted from an increased history of nausea episodes.

An outstanding chess player needs to accumulate massive visual and spatial information for chess configurations. Visual motion area (MT) is considered as a brain region specialized for visual motion perception and visuospatial attention processing. However, how long-term chess training shapes the functional connectivity patterns of MT, especially its functional subregions, has rarely been investigated. In our study, using resting-state functional connectivity (RSFC) and Granger causality analysis (GCA), we studied the changed functional couplings of MT subregions between 28 chess master players and 27 gender- and age-matched healthy novices to reveal the neural basis of long-term professional chess training. RSFC analysis identified decreased functional connections between right dorsal-anterior subregion (CI1.R) and left angular gyrus, and increased functional connections between right ventral-anterior MT subregion (CI2.R) and right superior temporal gyrus in chess experts. Moreover, GCA analyses further found increased mutual interactions of left angular gyrus and CI1.R in chess experts compared to novice players. These findings demonstrate that long-term professional chess training could enhance spatial perception and reconfiguration and semantic processing efficiency for superior performance.

Motion processing after sight restoration: No competition between visual recovery and auditory compensation

Purpose: To investigate the spatiotemporal evolution of cortical activation during the initiation of optokinetic nystagmus using magnetoencephalography. Background: Previous imaging studies of optokinetic nystagmus in humans using positron emission tomography and functional magnetic resonance imaging discovered activation of a large set of cortical and subcortical structures during steady-state optokinetic stimulation, but did not provide information on the temporal dynamics of the initial response. Imaging studies have shown that cortical areas responsible for vision in occipital and temporo-occipital areas are involved, i.e. cortical areas control optokinetic stimulation in humans. Magnetoencephalography provides measures that reflect neural ensemble activity in the millisecond time scale, allowing the identification of early cortical components of visuomotor integration. Design/Methods: We studied neuromagnetic cortical responses during the initiation of optokinetic nystagmus in 6 right-handed healthy subjects. Neuromagnetic activity was recorded with a whole-head magnetoencephalograph, consisting of 143 planar gradiometers. Results: The mean (±SD) latency between stimulus onset and initiation of optokinetic nystagmus was 177.7 ± 59 ms. Initiation of optokinetic nystagmus evoked an early component in the primary visual cortex starting at 40-90 ms prior to the onset of the slow phase of nystagmus. Almost simultaneously an overlapping second component occurred bilaterally in the temporo-occipital area (visual motion areas), pronounced in the right hemisphere, starting at 10-60 ms prior to the slow-phase onset. Both components showed long-duration activity lasting for up to 100 ms after slow-phase onset. Conclusions: Our findings suggest that the initiation of optokinetic nystagmus induces early cortical activation in the occipital cortex and almost simultaneously bilaterally in the temporo-occipital cortex. These cortical regions might represent essential areas for the monitoring of retinal slip.

This fMRI study of 24 healthy human participants investigated whether any part of the auditory cortex was more responsive to self-generated speech sounds compared to hearing another person speak. The results demonstrate a double dissociation in two different parts of the auditory cortex. In the right posterior superior temporal sulcus (RpSTS), activation was higher during speech production than listening to auditory stimuli, whereas in bilateral superior temporal gyri (STG), activation was higher for listening to auditory stimuli than during speech production. In the second part of the study, we investigated the function of the identified regions, by examining how activation changed across a range of listening and speech production tasks that systematically varied the demands on acoustic, semantic, phonological and orthographic processing. In RpSTS, activation during auditory conditions was higher in the absence of semantic cues, plausibly indicating increased attention to the spectral-temporal features of auditory inputs. In addition, RpSTS responded in the absence of any auditory inputs when participants were making one-back matching decisions on visually presented pseudowords. After analysing the influence of visual, phonological, semantic and orthographic processing, we propose that RpSTS (i) contributes to short term memory of speech sounds as well as (ii) spectral-temporal processing of auditory input and (iii) may play a role in integrating auditory expectations with auditory input. In contrast, activation in bilateral STG was sensitive to acoustic input and did not respond in the absence of auditory input. The special role of RpSTS during speech production therefore merits further investigation if we are to fully understand the neural mechanisms supporting speech production during speech acquisition, adult life, hearing loss and after brain injury.

Humans adeptly use visual motion to recognize socially-relevant facial information. The macaque provides a model visual system for studying neural coding of expression movements, as its superior temporal sulcus (STS) possesses brain areas selective for faces and areas sensitive to visual motion. We employed functional magnetic resonance imaging and facial stimuli to localize motion-sensitive areas (Mf areas), which responded more to dynamic faces compared to static faces, and face-selective areas, which responded selectively to faces compared to objects and places. Using multivariate analysis, we found that information about both dynamic and static facial expressions could be robustly decoded from Mf areas. By contrast, face-selective areas exhibited relatively less facial expression information. Classifiers trained with expressions from one motion type (dynamic or static) showed poor generalization to the other motion type, suggesting that Mf areas employ separate and non-confusable neural codes for dynamic and static presentations of the same expressions. We also show that some of the motion sensitivity elicited by facial stimuli was not specific to faces but could also be elicited by moving dots, particularly in FST and STPm/LST, confirming their already well-established low-level motion sensitivity. A different pattern was found in anterior STS, which responded more to dynamic than static faces but was not sensitive to dot motion. Overall, we show that emotional expressions are mostly represented outside of face-selective cortex, in areas sensitive to motion. These regions may play a fundamental role in enhancing recognition of facial expression despite the complex stimulus changes associated with motion.

To examine the impact of hearing loss (HL) on audiovisual (AV) processing in the aging population. We hypothesized that age-related HL would have a pervasive effect on sensory processing, extending beyond the auditory domain. Specifically, we predicted that decreased auditory input to the neural system, in the form of HL over time, would have deleterious effects on multisensory mechanisms. This study compared AV processing between older adults with normal hearing (N = 12) and older adults with mild to moderate sensorineural HL (N = 12). To do this, we recorded cortical evoked potentials that were elicited by watching and listening to recordings of a speaker saying the syllable "bi." Stimuli were presented in three conditions: when hearing the syllable "bi" (auditory), when viewing a person say "bi" (visual), and when seeing and hearing the syllables simultaneously (AV). Presentation level of the auditory stimulus was set to +30 dB SL for each listener to equalize auditory input across groups. In the AV condition, the normal-hearing group showed a clear and consistent decrease in P1 and N1 latencies as well as a reduction in P1 amplitude compared with the sum of the unimodal components (auditory + visual). These integration effects were absent or less consistent in HL participants. Despite controlling for auditory sensation level, visual influence on auditory processing was significantly less pronounced in HL individuals compared with controls, indicating diminished AV integration in this population. These results demonstrate that HL has a deleterious effect on how older adults combine what they see and hear. Although auditory amplification vastly improves the communication abilities in most hearing impaired individuals, the associated atrophy of multisensory mechanisms may contribute to a patient's difficulty in everyday settings. Our findings and related studies emphasize the potential value of multimodal tasks and stimuli in the assessment and rehabilitation of hearing impairments.

Cross-modal plasticity refers to the recruitment of cortical regions involved in the processing of one modality (e.g. vision) for processing other modalities (e.g. audition). The principles determining how and where cross-modal plasticity occurs remain poorly understood. Here, we investigate these principles by testing responses to auditory motion in visual motion area MT+ of congenitally blind and sighted individuals. Replicating previous reports, we find that MT+ as a whole shows a strong and selective responses to auditory motion in congenitally blind but not sighted individuals, suggesting that the emergence of this univariate response depends on experience. Importantly, however, multivoxel pattern analyses showed that MT+ contained information about different auditory motion conditions in both blind and sighted individuals. These results were specific to MT+ and not found in early visual cortex. Basic sensitivity to auditory motion in MT+ is thus experience-independent, which may be a basis for the region's strong cross-modal recruitment in congenital blindness.

Emotional context modulates embodied metaphor comprehension

V5/MT+ modulates spatio-temporal integration differently across and within hemifields: Causal evidence from TMS

We investigated whether previously reported differences between Alzheimer's disease and dementia with Lewy bodies (DLB) in resting occipital activity lead to activation differences within functionally specialized visual cortical areas and deactivation differences in the default network. Patients with Alzheimer's disease (n = 10; 5 male), DLB (n = 9; 4 male) and controls (n = 13; 5 male) performed three functional MRI (fMRI) scanning experiments involving visual colour, face or motion stimuli. Reaction time or accuracy in DLB and Alzheimer's disease differed significantly from controls but not between patient groups, with the exception of accuracy in the face task (DLB < Alzheimer's disease; P = 0.038). The most significant fMRI activations in the pooled data set were in left V4alpha for the colour task (Talairach coordinate: -30, -52, -24; P = 0.002 corrected), the right fusiform face area (FFA) for the face task (34, -48, -22; P = 0.005 corrected) and right intra-parietal sulcus (30, -66, 42; P = 0.003 corrected) for the motion task, with additional activity in right V5 (48, -64, 0; P = 0.015 corrected). Each task was associated with decreases in activity within the default network with prominent deactivation foci bilaterally in the posterior cingulate gyrus (+/-8, -48, 26; left P < 0.001; right P < 0.001 corrected) and medial frontal cortex (+/-18, 42, 32; left P < 0.001; right P < 0.001 corrected). Comparing patterns of task-related activity across groups, DLB patients showed more activation than Alzheimer patients within the superior temporal sulcus (STS) for the motion task (right STS: 44, 0, -20; P = 0.004 corrected; left STS: -40, -4, -26; P = 0.07 corrected). This difference could not be attributed to task performance, cognitive score or age [analysis of covariance (ANCOVA)F (2, 18) = 8.44, P = 0.003]. Within regions of interest, group activation differences were found for the face task (Alzheimer's disease > DLB P = 0.05; Alzheimer's disease > controls P = 0.14) and the motion task (DLB < Alzheimer's disease P = 0.031 and DLB < control P = 0.048). However, these differences could be explained by behavioural performance, failing to reach significance in the ANCOVA analysis. In the default network, group deactivation differences between controls and both patient groups were found for the colour and motion task (colour: control < Alzheimer's disease P = 0.02; control < DLB P = 0.019; motion: control < Alzheimer's disease P = 0.118; control < DLB P = 0.118) but could be accounted for by behavioural performance. The results suggest that cognitive fMRI can be used to detect both performance-dependent and performance-independent differences between Alzheimer's disease and DLB, reflecting the distribution of functional pathology in the two conditions.