A Spatiotemporal Perspective on Dynamical Computation in Neural Information Processing Systems.

Visual motion transforms visual space representations similarly throughout the human visual hierarchy

Self-motion perception relies primarily on the integration of the visual, vestibular, proprioceptive, and somatosensory systems. There is a gap in understanding how a temporal lag between visual and vestibular motion cues affects visual-vestibular weighting during self-motion perception. The beta band is an index of visual-vestibular weighting, in that robust beta event-related synchronization (ERS) is associated with visual weighting bias, and robust beta event-related desynchronization is associated with vestibular weighting bias. The present study examined modulation of event-related spectral power during a heading judgment task in which participants attended to either visual (optic flow) or physical (inertial cues stimulating the vestibular, proprioceptive and somatosensory systems) motion cues from a motion simulator mounted on a MOOG Stewart Platform. The temporal lag between the onset of visual and physical motion cues was manipulated to produce three lag conditions: simultaneous onset, visual before physical motion onset, and physical before visual motion onset. There were two main findings. First, we demonstrated that when the attended motion cue was presented before an ignored cue, the power of beta associated with the attended modality was greater than when visual-vestibular cues were presented simultaneously or when the ignored cue was presented first. This was the case for beta ERS when the visual-motion cue was attended to, and beta event-related desynchronization when the physical-motion cue was attended to. Second, we tested whether the power of feature-binding gamma ERS (demonstrated in audiovisual and visual-tactile integration studies) increased when the visual-vestibular cues were presented simultaneously versus with temporal asynchrony. We did not observe an increase in gamma ERS when cues were presented simultaneously, suggesting that electrophysiological markers of visual-vestibular binding differ from markers of audiovisual and visual-tactile integration. All event-related spectral power reported in this study were generated from dipoles projecting from the left and right motor areas, based on the results of Measure Projection Analysis.

Motion processing after sight restoration: No competition between visual recovery and auditory compensation

A convergent series of studies in monkeys and man suggests that the computation of visual space is performed in several brain regions for different behavioural purposes. Among these multiple spatial areas, the ventral intraparietal cortex, the putamen and the ventral aspect of the premotor cortex (area 6) contain a system for representing visual space near the face (peripersonal space). In these cerebral areas some neurons are bimodal: they have tactile receptive fields on the face, and they can also be driven by visual stimuli located near the tactile field. The spatial correspondence between the visual and tactile receptive fields provides a map of near visual space coded in body-part-centred co-ordinates. In the present study we demonstrate for the first time the existence of a visual peripersonal space centred on the face in humans. In patients with right hemispheric lesions, visual stimuli delivered in the space near the ipsilesional side of the face extinguished tactile stimuli on the contralesional side (cross-modal visuotactile extinction) to the same extent as did an ipsilesional tactile stimulation (unimodal tactile extinction). Furthermore, a visual stimulus presented in the proximity of the contralesional side of the face improved the detection of a left tactile stimulus: i.e. under bilateral tactile presentation patients were more accurate to report the presence of a left tactile stimulus when a simultaneous visual stimulus was presented near the left side of the face. However, when visual stimuli were delivered far from the face, visuotactile extinction and visuotactile facilitation effects were dramatically reduced. These findings are consistent with the hypothesis of a representation of visual peripersonal space coded in bodypart-centred co-ordinates, and they provide a striking demonstration of the modularity of human visual space.

Different studies have shown the efficiency of a feed-forward neural network in categorizing basic emotional facial expressions. However, recent findings in psychology and cognitive neuroscience suggest that visual recognition is not a pure bottom-up process but likely involves top-down recurrent connectivity. In the present computational study, we compared the performances of a pure bottom-up neural network (a standard multi-layer perceptron, MLP) with a neural network involving recurrent top-down connections (a simple recurrent network, SRN) in the anticipation of emotional expressions. In two complementary simulations, results revealed that the SRN outperformed the MLP for ambiguous intensities in the temporal sequence, when the emotions were not fully depicted but when sufficient contextual information (related to previous time frames) was provided. Taken together, these results suggest that, despite the cost of recurrent connections in terms of energy and processing time for biological organisms, they can provide a substantial advantage for the fast recognition of uncertain visual signals.

Visual motion pattern extraction and fusion for collision detection in complex dynamic scenes

Whether the auditory and visual systems use a similar coding strategy to represent motion direction is an open question. We investigated this question in the barn owl's optic tectum (OT) testing stimulus-specific adaptation (SSA) to the direction of motion. SSA, the reduction of the response to a repetitive stimulus that does not generalize to other stimuli, has been well established in OT neurons. SSA suggests a separate representation of the adapted stimulus in upstream pathways. So far, only SSA to static stimuli has been studied in the OT. Here, we examined adaptation to moving auditory and visual stimuli. SSA to motion direction was examined using repeated presentations of moving stimuli, occasionally switching motion to the opposite direction. Acoustic motion was either mimicked by varying binaural spatial cues or implemented in free field using a speaker array. While OT neurons displayed SSA to motion direction in visual space, neither stimulation paradigms elicited significant SSA to auditory motion direction. These findings show a qualitative difference in how auditory and visual motion is processed in the OT and support the existence of dedicated circuitry for representing motion direction in the early stages of visual but not the auditory system.

The estimation of visual motion has long been studied as a paradigmatic neural computation, and multiple models have been advanced to explain behavioral and neural responses to motion signals. A broad class of models, originating with the Reichardt correlator model, proposes that animals estimate motion by computing a temporal cross-correlation of light intensities from two neighboring points in visual space. These models provide a good description of experimental data in specific contexts but cannot explain motion percepts in stimuli lacking pairwise correlations. Here, we develop a theoretical formalism that can accommodate diverse stimuli and behavioral goals. To achieve this, we treat motion estimation as a problem of Bayesian inference. Pairwise models emerge as one component of the generalized strategy for motion estimation. However, correlation functions beyond second order enable more accurate motion estimation. Prior expectations that are asymmetric with respect to bright and dark contrast use correlations of both even and odd orders, and we show that psychophysical experiments using visual stimuli with symmetric probability distributions for contrast cannot reveal whether the subject uses odd-order correlators for motion estimation. This result highlights a gap in previous experiments, which have largely relied on symmetric contrast distributions. Our theoretical treatment provides a natural interpretation of many visual motion percepts, indicates that motion estimation should be revisited using a broader class of stimuli, demonstrates how correlation-based motion estimation is related to stimulus statistics, and provides multiple experimentally testable predictions.

Decision Making in Recurrent Neuronal Circuits

The backpropagation (BP)-type neural network (NN) was extended to deal with spatio-temporal input patterns, and such an NN model was trained to detect computer-simulated visual motion of light/dark boundaries (edges). The internal NN structure organized as a result of this learning was analyzed to find that, (1) one hidden unit is sufficient for uni-directional motion detection, (2) the spatio-temporal receptive field of the hidden unit exhibits stripes of excitatory and inhibitory areas parallel to a preferred direction, and this stripe-pattern receptive field works essentially as a 2D derivative filter, (3) after passing through this filter, the motion signal is processed with a threshold element followed by a low-pass filtering for smoothing. Thus, the present motion detecting NN is a nonlinear system with a sandwich structure of differentiation, threshold and low-pass filtering, as it is comprised of neurons having center-surround opponent receptive fields and a threshold unit with time-delayed input.

Massive green tides caused by Ulva prolifera have annually occurred in the Yellow Sea since 2007, which has attracted much attention from the government and society. There are associations between the green tides in the Yellow Sea and social response in the social media (i.e., Sina Weibo), which are bidirectional and could be captured by the bidirectional neural network. For instance, how to detect daily U. prolifera green tides by fusing remote sensing data with social media data, and how to use the observed U. prolifera green tides to infer the social response are two challenges of State Oceanic Administration, China. This article first illustrated that there are bidirectional associations between green tides and Sina Weibo data. Then, this article introduced a bidirectional spatio-temporal associative memory neural network (BSAMNN) model for modeling this bidirectional association from the spatio-temporal perspective. BSAMNN first extracted six characteristics from green tides and nine characteristics from the social responses in 2016-2019. Second, these characteristics were split by year, and the characteristics in 2016-2018 were, respectively, put into the bidirectional associative memory neural network (BAM), which is a two-layer artificial neural network. Based on the BAM results and the observed data, the residual network was constructed. Third, BSAMNN extracted the spatio-temporal rules from the characteristics in 2016-2018 as the constraints through the mining rule algorithm and modify the results via sea surface wind and ocean surface current. Last, BSAMNN put the characteristics in 2019 into BAM and used the residual network to modify the results, which was constrained by the spatio-temporal rules. The feasibility and reliability of our approach were demonstrated by using the U. prolifera green tides in 2019. The average accuracy, false alarm rate, and missing alarm rate of BSAMNN results were 0.69, 0.25, and 0.31, respectively, which was 0.11 higher, 0.10 lower, and 0.11 lower than that of the traditional BAM. The results indicated that our method is an effective alternative of linking the U. prolifera green tides and its public sentiments on social media.

This paper concerns certain difficult problems in image processing and perception: neuro-computation of visual motion information. The first part of this paper deals with the spatial physiological integration by the figure-ground discrimination neural network in the visual system of the fly. We have outlined the fundamental organization and algorithms of this neural network, and mainly concentrated on the results of computer simulations of spatial physiological integration. It has been shown that the gain control mechanism , the nonlinearity of synaptic transmission characteristic , the interaction between the two eyes , and the directional selectivity of the pool cells play decisive roles in the spatial physiological integration. In the second part, we have presented a self-organizing neural network for the perception of visual motion by using a retinotopic array of Reichardt's motion detectors and Kohonen's self-organizing maps. It .has been demonstrated by computer simulations that the network is able to learn to solve the ambiguities given by local motion detection mechanism. The resultant self-organized configuration in the output layer is resembling direction selective columns which first appear in area MT of the primate visual system. It has been explored that the spatio-temporal coherences, mapping, cooperation, competition, and Hebb rule are the basic neural principles for visual motion perception.

Visual space perception has been a topic of sustained research since the nineteenth century. Much of this research into the geometry of visual space, however, required observers to make judgments about spatial relationships between isolated points in total darkness. While a sizeable number of previous investigations have now explored visual space perception in outdoor natural environments, nearly all of the previous investigations evaluating the curvature of visual space have utilized only small numbers of observers. In the current experiment, a large number (30) of observers adjusted triangular configurations of markers in an outdoor field until they appeared either as equilateral or right triangles in depth. There was a wide range of outcomes, such that the observers’ judgments were consistent with elliptic, Euclidean, and hyperbolic geometry. There is thus no single consistent relationship between physical space and perceived space. Furthermore, the geometry of visual space frequently changes as the size of spatial configurations is varied—for many observers, judgments for small configurations are consistent with elliptic or Euclidean geometry while judgments for large configurations are frequently consistent with hyperbolic geometry.

Natural and Artificial Intelligence: A brief introduction to the interplay between AI and neuroscience research

Making Artificial Brains: Components, Topology, and Optimization.