Spatial and temporal contexts drive visual temporal integration via spatiotemporal normalisation

We have shown previously that amblyopia involves impaired detection of asynchrony between auditory and visual events. To distinguish whether this impairment represents a defect in temporal integration or nonintegrative multisensory processing (e.g., cross-modal matching), we used the temporal ventriloquism effect in which visual temporal order judgment (TOJ) is normally enhanced by a lagging auditory click. Participants with amblyopia (n = 9) and normally sighted controls (n = 9) performed a visual TOJ task. Pairs of clicks accompanied the two lights such that the first click preceded the first light, or second click lagged the second light by 100, 200, or 450 ms. Baseline audiovisual synchrony and visual-only conditions also were tested. Within both groups, just noticeable differences for the visual TOJ task were significantly reduced compared with baseline in the 100- and 200-ms click lag conditions. Within the amblyopia group, poorer stereo acuity and poorer visual acuity in the amblyopic eye were significantly associated with greater enhancement in visual TOJ performance in the 200-ms click lag condition. Audiovisual temporal integration is intact in amblyopia, as indicated by perceptual enhancement in the temporal ventriloquism effect. Furthermore, poorer stereo acuity and poorer visual acuity in the amblyopic eye are associated with a widened temporal binding window for the effect. These findings suggest that previously reported abnormalities in audiovisual multisensory processing may result from impaired cross-modal matching rather than a diminished capacity for temporal audiovisual integration.

Temporal resolution and temporal integration in the auditory system, with their conflicting time constants, are often regarded as paradox phenomena. In this study a model is presented for the simulation of a number of temporal resolution experiments (e.g., gap detection, forward masking) and classical test tone integration data (cf. contributions by Dau et al.). The model consists of two major parts. A preprocessing stage that transforms the acoustic stimulus into an internal representation, simulating the peripheral processing in the auditory system. Processing errors are accounted for by adding internal noise which introduces detection limits. The second stage is an optimum detector to simulate a ‘central’ decision device. The preprocessing stage consists mainly of three subunits: a Gamma tone filterbank to simulate peripheral filtering; a set of adaptive feedback loops that derives an adapted envelope of every peripheral filter output; a modulation filterbank (per peripheral channel) that analyzes stimulus periodicities in these adapted representations. The model is able to simulate gap detection and test tone integration experiments quantitatively without changing its parameters. It can be shown that within this model the modulation filterbank is vital to achieve this performance. [Supported by the DFG (Az. Mu/1201).]

Phasic neurons typically fire only for a fast-rising input, say at the onset of a step current, but not for steady or slow inputs, a property associated with type III excitability. Phasic neurons can show extraordinary temporal precision for phase locking and coincidence detection. Exemplars are found in the auditory brain stem where precise timing is used in sound localization. Phasicness at the cellular level arises from a dynamic, voltage-gated, negative feedback that can be recruited subthreshold, preventing the neuron from reaching spike threshold if the voltage does not rise fast enough. We consider two mechanisms for phasicness: a low threshold potassium current (subtractive mechanism) and a sodium current with subthreshold inactivation (divisive mechanism). We develop and analyze three reduced models with either divisive or subtractive mechanisms or both to gain insight into the dynamical mechanisms for the potentially high temporal precision of type III-excitable neurons. We compare their firing properties and performance for a range of stimuli. The models have characteristic non-monotonic input-output relations, firing rate vs. input intensity, for either stochastic current injection or Poisson-timed excitatory synaptic conductance trains. We assess performance according to precision of phase-locking and coincidence detection by the models' responses to repetitive packets of unitary excitatory synaptic inputs with more or less temporal coherence. We find that each mechanism contributes features but best performance is attained if both are present. The subtractive mechanism confers extraordinary precision for phase locking and coincidence detection but only within a restricted parameter range when the divisive mechanism of sodium inactivation is inoperative. The divisive mechanism guarantees robustness of phasic properties, without compromising excitability, although with somewhat less precision. Finally, we demonstrate that brief transient inhibition if properly timed can enhance the reliability of firing.

Understanding the neural mechanisms of action potential generation is critical to establish the way neural circuits generate and coordinate activity. Accordingly, we investigated the dynamics of action potential initiation in the GABAergic thalamic reticular nucleus (TRN) using in vivo intracellular recordings in cats in order to preserve anatomically-intact axo-dendritic distributions and naturally-occurring spatiotemporal patterns of synaptic activity in this structure that regulates the thalamic relay to neocortex. We found a wide operational range of voltage thresholds for action potentials, mostly due to intrinsic voltage-gated conductances and not synaptic activity driven by network oscillations. Varying levels of synchronous synaptic inputs produced fast rates of membrane potential depolarization preceding the action potential onset that were associated with lower thresholds and increased excitability, consistent with TRN neurons performing as coincidence detectors. On the other hand the presence of action potentials preceding any given spike was associated with more depolarized thresholds. The phase-plane trajectory of the action potential showed somato-dendritic propagation, but no obvious axon initial segment component, prominent in other neuronal classes and allegedly responsible for the high onset speed. Overall, our results suggest that TRN neurons could flexibly integrate synaptic inputs to discharge action potentials over wide voltage ranges, and perform as coincidence detectors and temporal integrators, supported by a dynamic action potential threshold.

It has been suggested that perception and action can be understood as evolving in temporal epochs or sequential processing units. Successive events are fused into units forming a unitary experience or “psychological present.” Studies have identified several temporal integration levels on different time scales which are fundamental for our understanding of behavior and subjective experience. In recent literature concerning the philosophy and neuroscience of consciousness these separate temporal processing levels are not always precisely distinguished. Therefore, empirical evidence from psychophysics and neuropsychology on these distinct temporal processing levels is presented and discussed within philosophical conceptualizations of time experience. On an elementary level, one can identify a functional moment, a basic temporal building block of perception in the range of milliseconds that defines simultaneity and succession. Below a certain threshold temporal order is not perceived, individual events are processed as co-temporal. On a second level, an experienced moment, which is based on temporal integration of up to a few seconds, has been reported in many qualitatively different experiments in perception and action. It has been suggested that this segmental processing mechanism creates temporal windows that provide a logistical basis for conscious representation and the experience of nowness. On a third level of integration, continuity of experience is enabled by working memory in the range of multiple seconds allowing the maintenance of cognitive operations and emotional feelings, leading to mental presence, a temporal window of an individual’s experienced presence.

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, ‘Geometrical’ and, in particular, ‘Tilt Illusions’ are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. The model is a variation of classical receptive field implementation for simple cells in early stages of vision with the scales tuned to the object/texture sizes in the pattern. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as ‘Anchoring theory’ and ‘Perceptual grouping’.

The present study investigated whether and to which extent temporal integration in bats is influenced by echolocation behavior. One way to quantify temporal integration is to measure the detection threshold for a pair of short tone pips as a function of the temporal separation between the pips. To asses the effect of preceding sonar emission on temporal integration in the bat, Megaderma lyra, the detection thresholds of identical subjects were measured in a passive as well as in an active paradigm. In the passive paradigm, the presentation of the pip pairs was independent of the bats' sonar emissions; in the active paradigm, the presentation was triggered by the bats' sonar emissions. In both cases, the bats showed a very short integration time in the range of 100-200 micros. Moreover, the comparison of the active and passive results within each bat revealed no systematic differences in the two measuring paradigms. These results indicate that temporal integration is not influenced by echolocation. Simulations with a computer model of cochlear filtering based on measurements of M. lyra cochlear tuning suggest that the perceptual temporal integration is dominated by the integration of the cochlear filters.

The physiological optics and neurology of vision are topics with much breadth and perhaps even more depth. Though many people have a cursory understanding of the visual system, there are some less-commonly known aspects that are vitally important to virtual environments research and practice. In this paper, we present an elementary discussion of the optical and neural elements involved in the early stages of vision and their relationship to virtual environments.

32 мая 1773 года Герцог из любимого фильма «Тот самый Мюнхгаузен» произнес знаменательные слова: «Поразительно... Поразительно, как наш народ гармонирует с природой!» Чуть позже его почти земляк Фриденсрайх Хундертвассер заявил, что счастье – это слияние с природой. Можно счастливо жить без денег. Без природы – нельзя. Слияние с природой сыграло большую роль в жизни этого замечательного человека, и даже в его смерти. Мы уже почти научились жить без денег. Но без природы жить не хочется, да и невозможно. «Хотите об этом поговорить?» – психоаналитично спросили мы сами себя. И, как честные пациенты, ответили себе: «Да, хотим». В скудные кризисные годы наступает самое время для рефлексии, а рефлексировать лучше всего на природе. Так сложилась тема очередного фестиваля «Зодчество Восточной Сибири» (13–46, 72–81). В материалах номера продолжается сквозная тема города для людей. Не следует думать, что архитектор создает только здания. Не камень или его современные эрзацы, и даже не плющ, прикрывающий строительные неудачи, – материал зодчего. Алехандро Аравена, лауреат Притцкеровской премии за последний год (крайний, как сейчас принято говорить), утверждает: самое главное в урбанизме – общественные пространства. Переменчивые, разнообразные, наполненные смыслами и визуальными событиями, составляющие особую природу города. Как сочетать элементы этой «второй природы» с живыми биоценозами природы первой? И всегда ли можно отличить одно от другого? К 355-летию Иркутска по почетному поручению мэра ведущие иркутские архитекторы выполнили более двадцати проектов скверов, хороших и разных (92–112). Кроме будущих скверов и карманных парков Иркутска в номере представлены зеленые пространства Москвы, Токио, Лондона и Парижа, городов Германии, конкурсные проекты аквариума в Нью-Йорке и корейского конного парка. Объектом номера мы выбрали удивительный музей поля Куликова, созданный по проекту Сергея Гнедовского. Необычная архитектура роднит новый музей с тектоническим разломом земли от невыносимого напряжения в ожидании битвы (113–123). А Олег Явейн и Илья Лежава вернут нас к классику ХХ века – Мису ван дер Роэ и его уникальному диалогу с природой (162–175). Тема соотношения архитектуры и природы неисчерпаема и безгранична, как обе эти сущности. Поэтому в новом номере нашего журнала, как обычно, больше вопросов, чем ответов. Такова природа нашего журнала.

Light fields in the real world are highly complex because they contain multiple illumination sources, including indirect light reflected from other surfaces. Such interreflexions are well compensated by the human visual system, by employing prior knowledge about the nature of the light field (Mach-card experiment: Bloj et al., 1999). Interreflections are the rule in natural environments and therefore a general problem for all organisms relying on colour vision, e.g. bees. Honeybees have excellent colour constancy as demonstrated for flat Mondrian-like scenes (Werner et al., 1988). However, bees have far fewer neurons available for these operations than humans and it is therefore interesting to ask: how can a complex colour vision task, like compensating the effect of light fields in 3D scenes, be solved by a relatively simple system? I will present a Mach-card experiment with free-flying honeybees (apis mellifera), where individual bees were trained to discriminate between green/white cards folded in different spatial configurations: (1) planar (2) concave and (3) convex. The stimuli (paper cards, 2.5 x 2.5 x 6 cm) were presented in front of a vertical, "grey" background (a turntable disc, 80 cm diameter). Previous experiments had shown that bees are able to resolve the depth profile of similar 3D forms. The results show that the bees recognized a trained green/white pattern irrespective of its spatial configuration, in other words, bees compensated the occurring interreflexions. Further experiments, involving simulations of the interreflexions, showed that the compensation depends on the consistency of spatial and chromatic cues. This suggests that bees use their knowledge about the spatial configuration in 3D scenes in order to discount the effects of mutual illuminations. Whether this involves cognitive inference, as it suggested for human colour vision, or is driven by hard-wired interconnections, remains to be solved. Meeting abstract presented at VSS 2014

Visual cryptography scheme (VCS) is an image secret sharing method which exploits the spatial responds characteristics of Human visual system (HVS). Applications of traditional OR and XOR based VCSs are seriously limited by the implementation carrier in practice. In this paper, we proposed a new kind of VCS which is implemented on modern display terminal of high refresh rates. Our approach exploits the temporal responds characteristics of HVS that light signals are temporal integrated into a single steady continuous one if the frequency exceeds critical fusion frequency (CFF). Furthermore, basing on the proposed VCS, we implement an information security display technology that can prevent unauthorized photography. Only authorized viewers can recover the secret information with the help of synchronized glass. While unauthorized viewers with naked eye or camera get nothing about the secret information. Experimental results show the effectiveness of proposed temporal integration based VCS and information security display technology.

To rapidly process biologically relevant stimuli, sensory systems have developed a broad variety of coding mechanisms like parallel processing and coincidence detection. Parallel processing (e.g., in the visual system), increases both computational capacity and processing speed by simultaneously coding different aspects of the same stimulus. Coincidence detection is an efficient way to integrate information from different sources. Coincidence has been shown to promote associative learning and memory or stimulus feature detection (e.g., in auditory delay lines). Within the dual olfactory pathway of the honeybee both of these mechanisms might be implemented by uniglomerular projection neurons (PNs) that transfer information from the primary olfactory centers, the antennal lobe (AL), to a multimodal integration center, the mushroom body (MB). PNs from anatomically distinct tracts respond to the same stimulus space, but have different physiological properties, characteristics that are prerequisites for parallel processing of different stimulus aspects. However, the PN pathways also display mirror-imaged like anatomical trajectories that resemble neuronal coincidence detectors as known from auditory delay lines. To investigate temporal processing of olfactory information, we recorded PN odor responses simultaneously from both tracts and measured coincident activity of PNs within and between tracts. Our results show that coincidence levels are different within each of the two tracts. Coincidence also occurs between tracts, but to a minor extent compared to coincidence within tracts. Taken together our findings support the relevance of spike timing in coding of olfactory information (temporal code).

Colour information is processed by many stages in the visual system. Primate colour vision relies on three photoreceptors, cones, which sample visible light and send signals to the second stage, cone–opponent units. Surprisingly this stage determines not only the threshold detection for chromatic patches, but also matching surface colours under various illuminations. Hue discrimination at detection thresholds reveals contribution of the third stage, colour-opponency, which determines colour categories, or unique hues. These hues remain constant for para-central, peripheral field locations, providing the reference for veridical vision. However, more challenging tasks of colour identification require contributions from higher colour centres (V4–complex, TEO, IT).The aim of this study is to present key experiments illustrating the contributions of the cone opponent stage and unique hues in low-level colour vision, whereas the effect of damage to various stages will be discussed in terms of the distributed representations in the visual system.The experiments showed that the colour apparatus uses signals derived from the entire visual field (i.e. panoramic viewing), perhaps using some form of spatial and temporal integration. This observation is supported by the fact that when a matching paradigm is used to compare peripheral and central colour samples, peripherally invariant hues correspond to unique hues. And, finally, there is a trade-off between sensitivity and selectivity, and it seems likely that neurons form flexible assemblies which can act together to extract either hue, giving fine hue discrimination but poor sensitivity, or extract contrast giving high chromatic sensitivity but having poor hue discrimination.

Previously we showed that network-based modelling of brain connectivity interacts strongly with the shape and exact location of brain regions, such that cross-subject variations in the spatial configuration of functional brain regions are being interpreted as changes in functional connectivity (Bijsterbosch et al., 2018). Here we show that these spatial effects on connectivity estimates actually occur as a result of spatial overlap between brain networks. This is shown to systematically bias connectivity estimates obtained from group spatial ICA followed by dual regression. We introduce an extended method that addresses the bias and achieves more accurate connectivity estimates.

Robust sound source localization is performed by the human auditory system even in challenging acoustic conditions and in previously unencountered, complex scenarios. Here a computational binaural localization model is proposed that possesses mechanisms for handling of corrupted or unreliable localization cues and generalization across different acoustic situations. Central to the model is the use of interaural coherence, measured as interaural vector strength (IVS), to dynamically weight the importance of observed interaural phase (IPD) and level (ILD) differences in frequency bands up to 1.4 kHz. This is accomplished through formulation of a probabilistic model in which the ILD and IPD distributions pertaining to a specific source location are dependent on observed interaural coherence. Bayesian computation of the direction-of-arrival probability map naturally leads to coherence-weighted integration of location cues across frequency and time. Results confirm the model's validity through statistical analyses of interaural parameter values. Simulated localization experiments show that even data points with low reliability (i.e., low IVS) can be exploited to enhance localization performance. A temporal integration length of at least 200 ms is required to gain a benefit; this is in accordance with previous psychoacoustic findings on temporal integration of spatial cues in the human auditory system.