Laboratoire des Systèmes Perceptifs

Abstract The visual system is known to integrate sensory information across both space and time, but these dimensions are often studied separately. Here we provide novel behavioural evidence for an early integration over space and time, and propose a computational model of this integration that is biologically plausible. We designed a controlled task that forces human participants to integrate information over both space and time to perform well. Stimuli were temporally segmented rings of oriented elements and participants had to detect a missing element in the ring. We found that temporal integration was enhanced when successive visual events were temporally proximate, spatially aligned to form a continuous contour, but also, and more surprisingly, when they were brief. Notably, the spatial effects on integration are temporally asymmetric: integration is facilitated when more events are presented earlier in the sequence, and even more so when they form a collinear contour. These findings indicate that temporal integration is sensitive to both the spatial configuration and the number of visual events presented over time, supporting a continuous rather than discrete process at early visual stages. All of these results are well accounted for by a computational model in which temporal integration arises from the overlap of internal signals generated by individual visual events. These signals are shaped by a spatially and temporally tuned divisive normalisation mechanism and integrated via coincidence detection. Predictions from this model match human performance in seminal past work as well as novel stimulus displays, offering an account of how spatio-temporal interactions determine what we perceive and when.

While deep learning has enabled the decoding of language from intracranial brain recordings, achieving this with non-invasive recordings remains an open challenge. We introduce a deep learning pipeline to decode individual words from electro- (EEG) and magneto-encephalography (MEG) signals. We evaluate our approach on seven public datasets and two datasets which we collect ourselves, amounting to a total of 723 participants reading or listening to five million words in three languages. Our model outperforms existing methods consistently across participants, devices, languages, and tasks, and can decode words absent from the training set. Our analyses highlight the importance of the recording device and experimental protocol: MEG and reading are easier to decode than EEG and listening, and decoding performance consistently increases with the amount of data used for training and for averaging during testing. Overall, our findings delineate the path and remaining challenges towards building non-invasive brain decoders for natural language.

The visual systems of animals work in diverse and constantly changing environments where organism survival requires effective senses. To study the hierarchical brain networks that perform visual information processing, vision scientists require suitable tools, and Motion Clouds (MCs)-a dense mixture of drifting Gabor textons-serve as a versatile solution. Here, we present an open toolbox intended for the bespoke use of MC functions and objects within modeling or experimental psychophysics contexts, including easy integration within Psychtoolbox or PsychoPy environments. The toolbox includes output visualization via a Graphic User Interface. Visualizations of parameter changes in real time give users an intuitive feel for adjustments to texture features like orientation, spatiotemporal frequencies, bandwidth, and speed. Vector calculus tools serve the frame-by-frame autoregressive generation of fully controlled stimuli, and use of the GPU allows this to be done in real time for typical stimulus array sizes. We give illustrative examples of experimental use to highlight the potential with both simple and composite stimuli. The toolbox is developed for, and by, researchers interested in psychophysics, visual neurophysiology, and mathematical and computational models. We argue the case that in all these fields, MCs can bridge the gap between well- parameterized synthetic stimuli like dots or gratings and more complex and less controlled natural videos.

When visual objects strike our retinas, they trigger a cascade of activity of different durations and delays along the visual processing hierarchy, making it difficult to predict exactly when they are perceived (Nishida & Johnston, 2002; Curr. Biol.). Here we investigate which temporal features of the visual stimulus are used to infer when an event is perceived. We presented two events in succession, each event being a pair of Gaussian blobs (1 dva) located at a fixed eccentricity on either side of fixation. We varied the delay between the events (800 or 1200 ms), and participants were instructed to reproduce this delay by pressing a key at the appropriate time after the second event. The orientation of the blobs was random for the first pair, and rotated by 120 degrees for the second. We manipulated the temporal profiles of the events to investigate when they are perceived. Contrast of the first event followed a Gaussian modulation. Contrast of the second event had either a Gaussian, or an asymmetrical temporal profile built from a weighted sum of two Gaussians shifted in time by 100 ms. These stimuli allowed us to independently vary the times of the maximum and mean contrasts. We found a strong effect of the temporal contrast modulation on the perceived time. Depending on the direction of the contrast distribution’s skewness, the second pair was perceived earlier (~80 ms) or later (~30 ms) relative to a pair with a symmetrical Gaussian contrast distribution and identical maximum contrast time. From this pattern of results, the information used for estimating when an event occurred cannot be a threshold intensity or the time of maximum contrast. Instead, the time when integrated contrast reaches a threshold appears relevant for when an event occurred (Amano et al., 2006; J. Neurosci.).

Introduction: Homeostatic and Hebbian plasticity co-operate during the critical period, refining neuronal circuits, however the interaction between these two forms of plasticity is still unclear, especially in adulthood. Here we investigate directly this issue in adult humans using two consolidated paradigms to elicit each form of plasticity: the LTP-like change of the visual evoked potential (Kirk et al 2021) and the shift of ocular dominance induced by short-term monocular deprivation (Baroncelli & Lunghi 2021). Methods: We tested three different conditions in a group (N=17) of adult volunteers (counterbalanced order). In the homeostatic condition, we measured ocular dominance (binocular rivalry between gratings, 2°, 2cpd, 64% contrast) before and after 1h of monocular deprivation (MD). In the Hebbian condition, we measured visual evoked potentials (VEP) in response to flashed (mean rate 1Hz) visual stimuli (checkerboard: 4°, 2 cpd, 64% contrast) presented binocularly in the upper visual field, before and after a 2-min high-frequency visual stimulation (HFS, checkerboard flickering at 9Hz). The mixed condition was similar to the homeostatic condition, except that HFS was delivered just before MD. Results: In the homeostatic and hebbian conditions we confirmed that the two experimental paradigms successfully induced each form of plasticity: ocular dominance shifted in favor of the deprived eye after MD (t(16)=8.23, p<0.001), and the amplitude of the N1b component of the VEP was reduced after HFS (t(16)=-2.569, p=0.021). Importantly, across participants, the effect of MD and the effect of HFS were correlated (r=-0.538, p=0.026). In the mixed condition, the effect of MD was comparable to the homeostatic condition (t(16)=-0.138, p=0.892). Conclusion: Homeostatic and Hebbian plasticity correlate in amplitude in adult humans, but they do not seem to interact when induced simultaneously. The relationship between these two forms of plasticity can explain the success of homeostatic plasticity based paradigms for the treatment of adult amblyopia.

Humans acquire sensory information via fast, highly specialized detectors: For example, edge detectors monitor restricted regions of visual space over timescales of 100-200ms. Surprisingly, this study demonstrates that their operation is nevertheless shaped by the ecological consistency of slow global statistical structure in the environment. In the experiments, humans acquired feature information from brief localized elements embedded within a virtual environment. Cast shadows are important for determining the appearance and layout of the environment. When the statistical reliability of shadows was manipulated, human feature detectors implicitly adapted to these changes over minutes, adjusting their response properties to emphasize either "image-based" or "object-based" anchoring of local visual elements. More specifically, local visual operators were more firmly anchored around object representations when shadows were reliable. As shadow reliability was reduced, visual operators disengaged from objects and became anchored around image features. These results indicate that the notion of sensory adaptation must be reframed around complex statistical constructs with ecological validity. These constructs far exceed the spatiotemporal selectivity bandwidth of sensory detectors, thus demonstrating the highly integrated nature of sensory processing during natural behavior.

Abstract Enhancing speech perception in everyday noisy acoustic environments remains an outstanding challenge for hearing aids. Speech separation technology is improving rapidly, but hearing devices cannot fully exploit this advance without knowing which sound sources the user wants to hear. Even with high-quality source separation, the hearing aid must know which speech streams to enhance and which to suppress. Advances in EEG-based decoding of auditory attention raise the potential of neurosteering, in which a hearing instrument selectively enhances the sound sources that a hearing-impaired listener is focusing their attention on. Here, we present and discuss a real-time brain-computer interface (BCI) system that combines a stimulus-response model based on canonical correlation analysis (CCA) for real-time EEG attention decoding, coupled with a multi-microphone hardware platform enabling low-latency real-time speech separation through spatial beamforming. We provide an overview of the system and its various components, discuss prospects and limitations of the technology, and illustrate its application with case studies of listeners steering acoustic feedback of competing speech streams via real-time attention decoding. A software implementation code of the system is publicly available for further research and explorations.

In this study, we investigated the effect of specific noise realizations on the discrimination of two consonants, /b/ and /d/. For this purpose, we collected data from twelve participants, who listened to /aba/ or /ada/ embedded in one of three background noises. All noises had the same long-term spectrum but differed in the amount of random envelope fluctuations. The data were analyzed on a trial-by-trial basis using the reverse-correlation method. The results revealed that it is possible to predict the categorical responses with better-than-chance accuracy purely based on the spectro-temporal distribution of the random envelope fluctuations of the corresponding noises, without taking into account the actual targets or the signal-to-noise ratios used in the trials. The effect of the noise fluctuations explained on average 8.1% of the participants' responses in white noise, a proportion that increased up to 13.3% for noises with a larger amount of fluctuations. The estimated time-frequency weights revealed that the measured effect originated from confusions between noise fluctuations and relevant acoustic cues from the target sounds. Similar conclusions were obtained from simulations using an artificial listener.