Rapid Temporal Recalibration to Audiovisual Asynchrony Occurs Across the Difference in Neural Processing Speed Based on Spatial Frequency.

In natural scenes, audiovisual events deriving from the same source are synchronized at their origin. However, from the perspective of the observer, there are likely to be significant multisensory delays due to physical and neural latencies. Fortunately, our brain appears to compensate for the resulting latency differences by rapidly adapting to asynchronous audiovisual events by shifting the point of subjective synchrony (PSS) in the direction of the leading modality of the most recent event. Here we examined whether it is the perceived modality order of this prior lag or its physical order that determines the direction of the subsequent rapid recalibration. On each experimental trial, a brief tone pip and flash were presented across a range of stimulus onset asynchronies (SOAs). The participants' task alternated over trials: On adaptor trials, audition either led or lagged vision with fixed SOAs, and participants judged the order of the audiovisual event; on test trials, the SOA as well as the modality order varied randomly, and participants judged whether or not the event was synchronized. For test trials, we showed that the PSS shifted in the direction of the physical rather than the perceived (reported) modality order of the preceding adaptor trial. These results suggest that rapid temporal recalibration is determined by the physical timing of the preceding events, not by one's prior perceptual decisions.

The red-green axis of colour vision evolved recently in primate evolutionary history. Signals serving red-green colour vision travel together with signals serving spatial vision, in the parvocellular (PC) division of the subcortical visual pathway. However, the question of whether receptive fields of PC pathway cells are specialized to transmit red-green colour signals remains unresolved. We addressed this question in single-cell recordings from the lateral geniculate nucleus of anaesthetized marmosets. Marmosets show a high proportion of dichromatic (red-green colour-blind) individuals, allowing spatial and colour tuning properties of PC cells to be directly compared in dichromatic and trichromatic visual systems. We measured spatial frequency tuning for sine gratings that provided selective stimulation of individual photoreceptor types. We found that in trichromatic marmosets, the foveal visual field representation is dominated by red-green colour-selective PC cells. Colour selectivity of PC cells is reduced at greater eccentricities, but cone inputs to centre and surround are biased to create more selectivity than predicted by a purely 'random wiring' model. Thus, one-to-one connections in the fovea are sufficient, but not necessary, to create colour-selective responses. The distribution of spatial tuning properties for achromatic stimuli shows almost complete overlap between PC cells recorded in dichromatic and trichromatic marmosets. These data indicate that transmission of red-green colour signals has been enabled by centre-surround receptive fields of PC cells, and has not altered the capacity of PC cells to serve high-acuity vision at high stimulus contrast.

Temporal and spatial frequency tuning of the flicker motion aftereffect.

ABSTRACTCurrent models of visual perception suggest that, during scene categorization, low spatial frequencies (LSF) are rapidly processed and activate plausible interpretations of visual input. This coarse analysis would be used to guide subsequent processing of high spatial frequencies (HSF). The present study aimed to further examine how information from LSF and HSF interact and influence each other during scene categorization. In a first experimental session, participants had to categorize LSF and HSF filtered scenes belonging to two different semantic categories (artificial vs. natural). In a second experimental session, we used hybrid scenes as stimuli made by combining LSF and HSF from two different scenes which were semantically similar or dissimilar. Half of the participants categorized LSF scenes in hybrids, and the other half categorized HSF scenes in hybrids. Stimuli were presented for 30 or 100 ms. Session 1 results showed better performance for LSF than HSF scene categorization. Session 2 scene categorization was faster when participants attended and categorized LSF than HSF scene in hybrids. The semantic interference of a semantically dissimilar HSF scene on LSF scene categorization was greater than the semantic interference of a semantically dissimilar LSF scene on HSF scene categorization, irrespective of exposure duration. These results suggest a LSF advantage for scene categorization, and highlight the prominent role of HSF information when there is uncertainty about the visual stimulus, in order to disentangle between alternative interpretations.

Virtual reality (VR) or augmented reality (AR) games using head-mounted displays (HMDs) are becoming increasingly popular in recent games. These games can present wider visual stimuli than TVs or handheld games. Moreover, the location of auditory stimuli is presented to the same location as visual stimuli. Therefore, we propose presenting visual stimuli to auditory stimuli rather than presenting auditory stimuli to visual stimuli. When we present visual stimuli to auditory stimuli, it is necessary to clarify how far the locations of the sound source and visual stimuli can be shifted. Thus, we examined varying degrees of spatial disparity between auditory and visual stimuli to determine whether they are still perceived as originating from the same location. The ventriloquism effect is known as a cross-modality between the locations of auditory and visual stimuli. Many researchers investigated the ventriloquism effect; however, there is no research on the effect of visibility of a loudspeaker playing a sound on the ventriloquism effect. In this study, we aim to clarify the effect of visibility of a loudspeaker playing a sound on the ventriloquism effect. For this purpose, we conducted two experiments to determine whether auditory and visual stimuli are originating from the same location when there are varying degrees of spatial disparity between them and measure their angle of origin. One was an AR condition experiment in which measurements were made with the loudspeaker visible, whereas the other was a VR condition experiment in which the loudspeaker was not visible. From the experimental results, the discrimination threshold of angle was more significant under the condition in which the loudspeaker was visible (AR condition experiment) than under the condition in which the loudspeaker was not visible (VR condition experiment). The results show that the ventriloquism effect is more substantial when the loudspeaker is visible.

People with schizophrenia demonstrate perceptual organization impairments, and these are thought to contribute to their face processing difficulties. We examined the neural substrates of emotionally neutral face processing in schizophrenia by investigating neural activity under three stimulus conditions: faces characterized by the full spectrum of spatial frequencies, faces with low spatial frequency information removed [high spatial frequency (HSF) condition], and faces with high spatial frequency information removed [low spatial frequency (LSF) condition]. Face perception in the HSF condition is more reliant on local feature processing whereas perception in the LSF condition requires greater reliance on global form processing. Past studies of perceptual organization in schizophrenia indicate that patients perform relatively more poorly with degraded stimuli but also that, when global information is absent, patients may perform better than controls because of their relatively increased ability to initially process individual features. Therefore, we hypothesized that people with schizophrenia (n=14) would demonstrate greater face processing difficulties than controls (n=13) in the LSF condition, whereas they would demonstrate a smaller difference or superior performance in the HSF condition. In a gender-discrimination task, behavioral data indicated high levels of accuracy for both groups, with a trend toward an interaction involving higher patient performance in the HSF condition and poorer patient performance in the LSF condition. Patients demonstrated greater activity in the fusiform gyrus compared to controls in both degraded conditions. These data suggest that impairments in basic integration abilities may be compensated for by relatively increased activity in this region.

Influence of spatial frequency and emotion expression on face processing in patients with panic disorder

Glare visual acuity and contrast sensitivity are important indicators of visual quality. Studies have shown that the glare visual acuity and contrast sensitivity in dry eye patients tend to degenerate, further affecting their quality of life. The objective of this study was to investigate the effect of notch filters on glare VA and contrast sensitivity in patients with dry eye or with dry eye syndrome. 36 subjects in the 20‒65 age group were diagnosed as having dry eye disease or perceived dry eye syndromes themselves who were included after the initial screening with the OSDI questionnaire, and one was subsequently excluded as they had undergone retinal detachment surgery. Finally, 35 subjects (14 male and 21 female) with a mean age of 40.66 ± 15.62years participated in this study. All subjects wore their habitual prescriptions and four different filter lenses (namely 480, 620, dual 480 & 620 notch filter, and FL-41 tinted lens), and measured the parameters of glare visual acuity and contrast sensitivity using CSV-1000 and sine wave contrast test (SWCT), respectively. Student t-test and Repeated measurement analysis (R-ANOVA) were utilized by using SPSS 26.0 software. A dual-wavelength 480 & 620nm optical notch filter had a significant anti-glare effect decreasing glare disabilities or discomfort, and leading to better visual quality, the same effect was also shown on a 480nm notch filter lens. All participants showed a significant difference among the baseline, three notch filters (480nm, 620nm, dual-wavelength 480 & 620nm), and FL-41 tinted lens were used on SWCT_A (1.5cpd, F = 3.054, p = 0.019) and SWCT_E (18cpd, F = 2.840, p = 0.049); but did not show statistical different on SWCT_B (3cpd, F = 0.333, p = 0.771), SWCT_C (6cpd, F = 1.779, p = 0.159), and SWCT_D (12cpd, F = 1.447, p = 0.228). The baseline showed the best visual performance on CS at a low spatial frequency (SWCT_A, 1.5cpd), any filter might reduce the contrast sensitivity at low spatial frequencies in the clinical trial, whereas 480nm notch filter showed the best effectiveness on CS at a high spatial frequency (SWCT_E, 18cpd), the FL-41 lens that also filters out the 480nm band does not achieve the same effect. Moreover, patients with dry eye or those older than 40years old preferred optical multilayer notch filters to FL-41 tinted lenses. The 480- & 620-nm dual-wavelength and 480-nm single-wavelength notch filters have the best effect on the glare visual acuity and contrast sensitivity (CS) at high spatial frequencies in dry eye patients. The 620-nm notch filter performs better in CS at low and mid-low spatial frequencies; the FL-41 tinted lens performs poorly for glare VA and CS spatial frequencies examination. Patients with glare disabilities or CS disturbance at high spatial frequencies may choose a 480-nm notch filter lens, and patients who have CS disturbance at low spatial frequencies may consider a 620-nm notch filter for the prescription.

Increased visual task difficulty enhances attentional capture by both visual and auditory distractor stimuli

How multisensory cues affect processing in early sensory brain areas is not well understood. The inferior colliculus (IC) is an early auditory structure that is visually responsive (Porter KK, Metzger RR, Groh JM. Proc Natl Acad Sci USA 104: 17855-17860, 2007; Bulkin DA, Groh JM. Front Neural Circuits 6: 61, 2012; Bulkin DA, Groh JM. J Neurophysiol 107: 785-795, 2012), but little is known about how visual signals affect the IC's auditory representation. We explored how visual cues affect both spiking and local field potential (LFP) activity in the IC of two monkeys performing a task involving saccades to auditory, visual, or combined audiovisual stimuli. We confirm that LFPs are sensitive to the onset of fixation lights and the onset of visual targets presented during steady fixation. The LFP waveforms evoked by combined audiovisual stimuli differed from those evoked by sounds alone. In single-unit spiking activity, responses were weak when visual stimuli were presented alone, but visual stimuli could modulate the activity evoked by sounds in a stronger way. Such modulations could involve either increases or decreases in activity, and whether increases or decreases were observed was variable and not obviously correlated with the responses evoked by visual or auditory stimuli alone. These findings indicate that visual stimuli shape the IC's auditory representation in flexible ways that differ from those observed previously in multisensory areas.NEW & NOTEWORTHY We find that the inferior colliculus, a primarily auditory brain area, displays distinct population-level responses to visual stimuli. We also find that visual cues can influence the auditory responses of individual neurons. Together, the results provide insight into how relatively early sensory areas may play a role in combining multiple sensory modalities to refine the perception of complex environments.

In three experiments we measured reaction times (RTs) and error rates in identifying the global structure of spatially filtered stimuli whose spatial-frequency content was selected by means of three types of 2-D isotropic filters (Butterworth of order 2, Butterworth of order 10, and a filters with total or partial Gaussian spectral profile). In each experiment, low-pass (LP), bandpass (BP), and high-pass (HP) filtered stimuli, with nine centre or cut-off spatial frequencies, were used. Irrespective of the type of filter, the experimental results showed that: (a) RTs to stimuli with low spatial frequencies were shorter than those to stimuli with medium or high spatial frequencies, (b) RTs to LP filtered stimuli were nearly constant, but they increased in a nonmonotonic way with the filter centre spatial frequency in BP filtered stimuli and with the filter cut-off frequency in HP filtered stimuli, and (c) the identification of the global pattern occurred with all visible stimuli used, including BP and HP images without low spatial frequencies. To remove the possible influence of the energy, a fourth experiment was conducted with Gaussian filtered stimuli of equal contrast power (c(rms) = 0.065). Similar results to those described above were found for stimuli with spatial-frequency content higher than 2 cycles deg(-1). A model of isotropic first-order visual channels collecting the stimulus spectral energy in all orientations explains the RT data. A subsequent second-order nonlinear amplitude demodulation process, applied to the output of the most energetic first-order channel, could explain the perception of global structure of each spatially filtered stimulus, including images lacking low spatial frequencies.

The current studies indicated that sound can improve the visual perception. However, whether the gain is influenced by temporal disparity between visual and auditory stimuli, and whether the gain is the same for older and younger adults remains unclear. To clarify this mechanism, ten older adults and 10 younger adults were invited to confirm and visual/auditory discrimination task. The visual target task, visual irrelevant task, auditory target task, and auditory irrelevant task randomly presented on the right or left hemi-space of the central fixation point and the subjects were instructed to respond to target stimuli rapidly and accurately. The results showed that in the younger adults, the gain was greatest in the simultaneous audiovisual condition. With the enlargement of temporal disparity between visual stimulus and auditory stimulus, the gain decreased significantly, and the alterative slope was different between auditory proceeded visual stimulus and visual proceeded auditory stimulus. In the older adults, the variation tendency was similar to younger adults, but in all conditions, the response of older adults was significantly slower than that of younger adults. Our results suggested that the visual gains were influenced by temporal disparity between auditory and visual stimuli, and further suggested that the gain mechanism between auditory proceeded visual stimuli and visual proceeded auditory stimuli was different, which indicated that the development of intelligent robot should consider the condition of the target subject to switch to the adapt procedure.

Streetscapes play a critical role in restorative landscapes, offering opportunities for promoting public well-being. Previous studies have predominantly examined the influence of visual and auditory stimuli on perceived restorativeness independently. There is a limited understanding of their interactive effects. In this research, 360 participants completed a series of experiments considering four distinct street types, including visual comfort assessment, acoustic environment assessment, and perceived restorativeness. They were assigned to a control group and one of three experimental groups, each receiving specific enhancement: visual stimuli, auditory stimuli, or a combination of audiovisual stimuli. The findings revealed that the experimental groups reported a greater sense of restorativeness compared to the control group. Notably, auditory stimuli demonstrated a more pronounced restorative effect than visual stimuli, while limited differences were found between auditory and audiovisual stimuli. The differences in experimental outcomes among the four street types are compared and discussed, highlighting context-specific guidelines for enhancing streetscape restorativeness. The research findings highlight enhancing the masking effect of soundscape in street environmental design. The study adds a novel multi-sensory approach to the current body of research on restorative landscapes, providing significant insights for the planning and design of streetscapes.

Eye movements transform a spatial scene into luminance modulations on the retina. Recent work has shown that this transformation is highly structured: within human temporal sensitivity, saccades deliver power that increases in proportion to spatial frequency (SF) up to a critical frequency and remains constant beyond that. Importantly, the critical SF increases with decreasing amplitude. Therefore, at sufficiently low SFs, larger saccades effectively deliver stronger input signals to the retina. Here we tested whether this input reformatting has the predicted perceptual consequences, by examining how large and small saccades (6o & 1o) affect contrast sensitivity. We measured relative sensitivity at two SFs: a reference (0.5 cpd), equal to the critical SF for the small saccade, and a probe at either a lower or higher SF (0.1/2.5 cpd). We predicted that large saccades enhance visibility only when the probe has a lower SF than the reference. Subjects (N=7) made instructed saccades while presented with a plaid of overlapping orthogonal gratings at the two SFs and reported which grating was more visible. Results closely follow theoretical predictions: psychometric functions following small and large saccades only differed with the lower SF probe, in which case the larger saccade significantly enhanced visibility. In sum, saccades enable selectivity not only in the spatial domain, but also in the spatial-frequency domain.

The barn owl's central auditory system creates a map of auditory space in the external nucleus of the inferior colliculus (ICX). Although the crucial role visual experience plays in the formation and maintenance of this auditory space map is well established, the mechanism by which vision influences ICX responses remains unclear. Surprisingly, previous experiments have found that in the absence of extensive pharmacological manipulation, visual stimuli do not drive neural responses in the ICX. Here we investigated the influence of dynamic visual stimuli on auditory responses in the ICX. We show that a salient visual stimulus, when coincident with an auditory stimulus, can modulate auditory responses in the ICX even though the same visual stimulus may elicit no neural responses when presented alone. For each ICX neuron, the most effective auditory and visual stimuli were located in the same region of space. In addition, the magnitude of the visual modulation of auditory responses was dependent on the context of the stimulus presentation with novel visual stimuli eliciting consistently larger response modulations than frequently presented visual stimuli. Thus the visual modulation of ICX responses is dependent on the characteristics of the visual stimulus as well as on the spatial and temporal correspondence of the auditory and visual stimuli. These results demonstrate moment-to-moment visual enhancements of auditory responsiveness that, in the short-term, increase auditory responses to salient bimodal stimuli and in the long-term could serve to instruct the adaptive auditory plasticity necessary to maintain accurate auditory orienting behavior.