Neural Network Model for Extracting Optic Flow

GENERAL COMMENTARY article Front. Hum. Neurosci., 06 October 2014Sec. Cognitive Neuroscience Volume 8 - 2014 | https://doi.org/10.3389/fnhum.2014.00786

When we travel in an environment, we have an optic flow on the retina. Neurons in the area MST of macaque monkeys are reported to analyze optic flow. The MST cells respond selectively to rotation, expansion/contraction and spiral motion. Previously, it was suggested that vector-field calculus is useful for analyzing optic flow. However, few MST models have been proposed so far, based on it. This paper adopts the vector-field hypothesis and explicitly proposes a novel neural network model for MST. Our model consists of hierarchically connected layers, namely, retina, V1, MT and MST. V1 cells measure local velocity. MT cells extract relative velocity with their antagonistic networks. MST cells simply collect the signals from MT cells and respond selectively to various types of optic flows. We demonstrate through a computer simulation that the simple architecture of our model is enough to explain a variety of results of neurophysiological experiments.

Amyotrophic lateral sclerosis (ALS) is a fatal motor neuron disorder that causes progressive paralysis. L-Citrulline is a non-essential neutral amino acid produced by L-arginine via nitric oxide synthase (NOS). According to previous studies, the pathogenesis of ALS entails glutamate toxicity, oxidative stress, protein misfolding, and neurofilament disruption. In addition, L-citrulline prevents neuronal cell death in brain ischemia; therefore, we investigated the change in the transport of L-citrulline under various pathological conditions in a cell line model of ALS. We examined the uptake of [14C]L-citrulline in wild-type (hSOD1wt/WT) and mutant NSC-34/ SOD1G93A (MT) cell lines. The cell viability was determined via MTT assay. A transport study was performed to determine the uptake of [14C]L-citrulline. Quantitative real-time polymerase chain reaction (qRT-PCR) analysis was performed to determine the expression levels of rat large neutral amino acid transported 1 (rLAT1) in ALS cell lines. Nitric oxide (NO) assay was performed using Griess reagent. L-Citrulline had a restorative effect on glutamate induced cell death, and increased [14C]L-citrulline uptake and mRNA levels of the large neutral amino acid transporter (LAT1) in the glutamate-treated ALS disease model (MT). NO levels increased significantly when MT cells were pretreated with glutamate for 24 h and restored by co-treatment with L-citrulline. Co-treatment of MT cells with L-arginine, an NO donor, increased NO levels. NSC-34 cells exposed to high glucose conditions showed a significant increase in [14C]L-citrulline uptake and LAT1 mRNA expression levels, which were restored to normal levels upon co-treatment with unlabeled L-citrulline. In contrast, exposure of the MT cell line to tumor necrosis factor alpha, lipopolysaccharides, and hypertonic condition decreased the uptake significantly which was restored to the normal level by co-treating with unlabeled L-citrulline. L-Citrulline can restore NO levels and cellular uptake in ALS-affected cells with glutamate cytotoxicity, pro-inflammatory cytokines, or other pathological states, suggesting that L-citrulline supplementation in ALS may play a key role in providing neuroprotection.

The fly visual system, although tiny when compared to the mammalian visual system, can still perform highly sophisticated spatial tasks like collision avoidance, landing on objects, pursuit of prey, etc. Flies outperform human-made autonomous flying systems in solving such spatial tasks by a long way. This is partly due to their ability to perceive and respond to optical flow generated by motion in the environment. They are also known to take actions that alter the image flow on their eyes. Higher level neurons in the fly visual system respond to different types of complex optical flows due to rotation and translation, by pooling information from local motion detectors called the elementary motion detectors (EMDs) in the lower level. In this sense, neuronal responses (spikes) from these optical flow detectors in the fly carry highly encoded signals: a single spike can represent a complex dynamical pattern of movement in the visual field. In this paper, we investigate how such highly encoded signals can be interpreted and utilized in the fly's brain, while solely operating on the encoded internal spike patterns within their brain and no direct external (unencoded) sensory information, i.e. a form of grounding. With a computational model of the optical flow detectors based on those in the fly, we show that specific pattern of action (or coordinated motor output) is the only way that a fly can decode its internal spikes and generate meaningful, relevant behavior based on that.

BackgroundThe dorsal parietal visual system plays an important role in self-motion perception and spatial cognition. It also strongly responds to visual inputs from the lower visual field. Postural control is modified in a process called sensory reweighting based on the reliability of available sensory sources. The question of whether visual stimuli presented to either the lower or upper visual field affect postural control and sensory reweighting has not been resolved. Research questionDo visual stimuli presented to the lower and upper visual fields affect postural control and sensory reweighting? MethodsTwenty-nine healthy young adults participated in the study. Four conditions (full visual field, upper visual field, lower visual field, and no optic flow condition) were simulated in a VR environment using a head-mounted display. The optic flow stimuli used were swarms of small white spheres originating from the central point of the visual field, moving radially towards the periphery, and expanding across the scene. Participants were instructed to stand quietly for 50 s under each visual condition. Using force plate signals, we measured the center of pressure (COP) signal in the horizontal plane and calculated its 95 % ellipse area, root mean square (RMS) deviations, the mean velocity, and power spectral density (PSD). ResultsOptic flow in the full and lower visual fields produced significantly smaller 95 % ellipse area and RMS of COP in the anterior-posterior direction compared to optic flow in the upper visual field. Furthermore, the PSD of the lower frequency band (0–0.3 Hz) was decreased and that of higher frequency bands (0.3–1 Hz and 1–3 Hz) was increased for the lower compared to the upper visual field. SignificanceVisual feedback affects static postural control more when presented in the lower visual field compared to the upper visual field.

Avian flight is guided by optic flow-the movement across the retina of images of surfaces and edges in the environment due to self-motion. In all vertebrates, there is a short pathway for optic flow information to reach pre-motor areas: retinal-recipient regions in the midbrain encode optic flow, which is then sent to the cerebellum. One well-known role for optic flow pathways to the cerebellum is the control of stabilizing eye movements (the optokinetic response). However, the role of this pathway in controlling locomotion is less well understood. Electrophysiological and tract tracing studies are revealing the functional connectivity of a more elaborate circuit through the avian cerebellum, which integrates optic flow with other sensory signals. Here we review the research supporting this framework and identify the cerebellar output centres, the lateral (CbL) and medial (CbM) cerebellar nuclei, as two key nodes with potentially distinct roles in flight control. The CbM receives bilateral optic flow information and projects to sites in the brainstem that suggest a primary role for flight control over time, such as during forward flight. The CbL receives monocular optic flow and other types of visual information. This site provides feedback to sensory areas throughout the brain and has a strong projection the nucleus ruber, which is known to have a dominant role in forelimb muscle control. This arrangement suggests primary roles for the CbL in the control of wing morphing and for rapid maneuvers.

Mice have a large visual field that is constantly stabilized by vestibular ocular reflex (VOR) driven eye rotations that counter head-rotations. While maintaining their extensive visual coverage is advantageous for predator detection, mice also track and capture prey using vision. However, in the freely moving animal quantifying object location in the field of view is challenging. Here, we developed a method to digitally reconstruct and quantify the visual scene of freely moving mice performing a visually based prey capture task. By isolating the visual sense and combining a mouse eye optic model with the head and eye rotations, the detailed reconstruction of the digital environment and retinal features were projected onto the corneal surface for comparison, and updated throughout the behavior. By quantifying the spatial location of objects in the visual scene and their motion throughout the behavior, we show that the prey image consistently falls within a small area of the VOR-stabilized visual field. This functional focus coincides with the region of minimal optic flow within the visual field and consequently area of minimal motion-induced image-blur, as during pursuit mice ran directly toward the prey. The functional focus lies in the upper-temporal part of the retina and coincides with the reported high density-region of Alpha-ON sustained retinal ganglion cells.

Animals and humans rely on optic flow to navigate cluttered and unknown environments. While most previous studies have focused on how organisms achieve self-motion perception through optic flow information, biological neural networks for navigation based on optic flow remain unexplored. Here, we propose a biologically plausible neural network model for optic flow-based reactive navigation. The model incorporates a primary visual cortex, which is responsible for generating a cortical representation of the optic flow field; a higher-order cortex, which calculates the focus of expansion (FOE) of the optic flow field; and a cerebellum, which generates motor commands. A feedback inhibitory pathway from V1 layer VI to layer IV is introduced, enhancing heading sensitivity and enabling rapid adaptation in dynamic environments. To achieve precise obstacle localization, we propose a dual encoding strategy that combines optic flow with depth maps derived from the optic flow field, FOE, and control acceleration. This strategy mitigates distortions in depth estimation near the expansion center and ensures more reliable obstacle representation. The cerebellum outputs motor commands for heading direction and speed control based on the output of the visual cortex. Simulations and real-world experiments with an intelligent vehicle confirm that the proposed model enables collision-free navigation across diverse scenarios and outperforms classical optic flow balance strategies in complex environments. These findings demonstrate that biologically inspired neural networks provide a feasible solution for visual reactive navigation in autonomous agents.

Event Abstract Back to Event Spatial vision in insects: How visual information processing is facilitated by an active flight and gaze strategy Martin Egelhaaf1*, Roland Kern1 and Jens P. Lindemann1 1 Bielefeld University, Neurobiology & CITEC, Germany Flying insects, such as blowflies and bees rely on the image flow on their eyes (“optic flow”) solving spatial tasks as diverse as collision avoidance and local navigation, i.e. localizing a previously learnt inconspicuous goal on the basis of landmark cues. In free flight flies and bees exhibit characteristic behavioural actions to shape the dynamics of the optic flow. By employing a saccadic flight and gaze strategy they squeeze rotations into brief and rapid turns leaving most of the flight time for various types of translational prototypical movements. Separation of rotational and translational optic flow components by a behavioural strategy is reasonable , since only the optic flow component induced by translations contains information about the relative distance of environmental structures from the animal. By combined electrophysiological analysis and model simulations we analyzed how during rotational and translational self-motion information about the spatial layout and the textural properties of both experimenter-designed and natural environments are represented at the different processing stages of the visual motion pathway. Only two major conclusions shall be mentioned here: (1) during intersaccadic translational flight, the mechanisms of local motion detection segregate in a computationally cheap way natural sceneries into their depth structure: nearby objects pop out in the activity profile of retinotopic arrays of local motion detectors, whereas more distant structures evoke only negligible responses. (2) Even after spatial pooling of local motion signals by the dendritic trees of the wide-field cells in the lobula complex of flies and bees, these distance dependent response modulations persist. Hence, these neurons, which are traditionally conceived as rotation sensors being involved in mediating compensatory optomotor responses, provide information about the spatial properties of the environment. In conclusion, the neural processing of spatial information in flies and bees is greatly facilitated by their saccadic flight and gaze strategy. This active vision strategy enables the nervous system to solve apparently complex spatial vision tasks in a computationally parsimonious way. Thus, by making use of the closed action–perception loop in a smart way, insects with their tiny brains are capable of performing extraordinarily well even on extremely fast timescales of some ten milliseconds. Acknowledgements The work in our lab is supported by the Deutsche Forschungsgemeinschaft (DFG) Keywords: motion vision, Spatial Vision, Optic Flow, collision avoidance, navigation, fly, Honeybee, Bumblebee, Flight control Conference: International Conference on Invertebrate Vision, Fjälkinge, Sweden, 1 Aug - 8 Aug, 2013. Presentation Type: Oral presentation preferred Topic: Motion vision Citation: Egelhaaf M, Kern R and Lindemann JP (2019). Spatial vision in insects: How visual information processing is facilitated by an active flight and gaze strategy. Front. Physiol. Conference Abstract: International Conference on Invertebrate Vision. doi: 10.3389/conf.fphys.2013.25.00044 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 25 Feb 2013; Published Online: 09 Dec 2019. * Correspondence: Prof. Martin Egelhaaf, Bielefeld University, Neurobiology & CITEC, Bielefeld, D-33501, Germany, martin.egelhaaf@uni-bielefeld.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Martin Egelhaaf Roland Kern Jens P Lindemann Google Martin Egelhaaf Roland Kern Jens P Lindemann Google Scholar Martin Egelhaaf Roland Kern Jens P Lindemann PubMed Martin Egelhaaf Roland Kern Jens P Lindemann Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

1. A study was made of the responses of single neurones in the inferior colliculus of bats to tone bursts with different rise times (or with different rates of amplitude increase) in order to determine whether the neurones were specialized for analysis of amplitude-modulated sound, especially the rising phase in amplitude.2. The response patterns of neurones which showed phasic on-responses usually did not change with rise time, although the response patterns of some neurones changed from phasic on-responses to inhibitory responses.3. The thresholds of responses to tone bursts increased when the rise time was lengthened. The amount of increase greatly differed from neurone to neurone. For the excitation of neurones which showed a large increase in threshold, the stimulus amplitude should quickly increase in amplitude. For tone bursts with a short rise time, some neurones showed an upper-threshold above which the sounds failed to excite them. The upper-threshold usually disappeared when the rise time was lengthened. For the excitation of neurones which showed an upper-threshold, the rate of amplitude increase and its extent were very important parameters.4. Lengthening the rise time of a tone burst from 0.5 to 98 msec caused various types of change in the excitatory area. In some cases, there was a diminution of the area and, in others, there was an expansion of the area. Neurones showing these changes may be considered to be specialized for responding to tone bursts with either a rapid or slow increase in amplitude.5. Changes with rise time occurred not only in the excitatory area, but also in the inhibitory area. The change in the inhibitory area of a given neurone, however, was not necessarily the same as that in the excitatory area.6. The latency of response usually changed as a function of stimulus amplitude and rise time. In 26% of the neurones studied, however, the latency was relatively constant regardless of stimulus amplitude and rise time. Some of the latency-constant neurones appeared to have properties suited for echo-ranging.7. The changes in excitatory area with rise time appear to involve neural inhibition and/or accommodation. Possible neural mechanisms for the changes in threshold, upper-threshold and response pattern and the properties of latency-constant neurones are discussed.

Optic flow-induced sway in anxiety disorders associated with space and motion discomfort

Adults with cortically-induced blindness (CB) affecting a quarter to a half of their visual field show greater variability in lane positioning when steering compared to those with intact vision. Because humans rely on visual information from optic flow to control steering, we hypothesized that these lane biases are caused in part by a disruption to motion processing caused by CB. To investigate, we examined the steering behavior of 21 CB drivers (11 left-sided, 10 right-sided visual deficits) and 9 visually intact controls in a closed-loop virtual reality steering task. Participants were instructed to maintain a central lane position while traveling at 19 m/s along a procedurally generated single-lane road. Turn direction (left/right) and turn radius (35m/55m/75m) varied between trials, and the quality of optic flow information was indirectly manipulated by altering the environmental texture density (low/medium/high). Right-sided CB participants maintained a similar average distance from the inner road edge as controls. Those with left-sided CB were less affected by changes in optic flow and turn direction. These differences were not explained by age, time since stroke, sparing of central vision, gaze direction, or saccade rate. Our results suggest that some left-sided CB participants place a lower weighting on optic flow information in the control of steering, possibly as a result of lateralization in the processing of motion. More broadly, our findings show that CB steering and gaze behavior are remarkably preserved despite the presence of visual deficits across large portions of the visual field.

Adults with cortically-induced blindness (CB) affecting a quarter to a half of their visual field show greater variability in lane positioning when driving compared to those with intact vision. Because humans rely on visual information from optic flow to control steering, we hypothesized that these lane biases are caused in part by a disruption to motion processing caused by CB. To investigate, we examined the steering behavior of 21 CB drivers (11 left-sided, 10 right-sided visual deficits) and 9 visually intact controls in a naturalistic virtual environment. Participants were instructed to maintain a central lane position while traveling at 19 m/s along a procedurally generated single-lane road. Turn direction (left/right) and turn radius (35m/55m/75m) varied between trials, and the quality of optic flow information was indirectly manipulated by altering the environmental texture density (low/medium/high). Right-sided CB participants maintained a similar average distance from the inner road edge as controls. Those with left-sided CB were less affected by changes in optic flow and turn direction. These differences were not explained by age, time since stroke, sparing of central vision, gaze direction, or saccade rate. Our results suggest that some left-sided CB participants place a lower weighting on optic flow information in the control of steering, possibly as a result of lateralization in the processing of motion. More broadly, our findings show that CB steering and gaze behavior are remarkably preserved despite the presence of visual deficits across large portions of the visual field.

A feedback control process based on self-motion perception contributes to postural stability; however, little is known about the visual modulation of postural muscles. The aim of this study was to investigate the effect of optic flow stimuli, presented full field, in the peripheral and foveal visual field, on muscular activation. Then, we assessed the correlation between optic flow, muscle activity and body sway in male and female subjects. We used surface electromyography (EMG) and stabilometry on 24 right-handed young adults. We recorded the bilateral activation of tibialis anterior, gastrocnemius medialis, biceps femoris and vastus medialis. EMG and center of pressure (COP) signals were acquired simultaneously. EMG signal amplitude was computed as root mean square normalized by baseline. We found a significant effect for muscles, gender and an interaction effect of muscle by gender (ANOVA, p < 0.001). Results showed different postural alignments in males and females. The COP spatial variability during peripheral stimuli was generally reduced. The prevalent direction of oscillation evoked by peripheral stimuli was clustered, while foveal and random stimuli induced distributed and randomized directions. Also for muscle activity, we found gender differences in the prevalent oscillation distributions evoked by optic flow. Visual stimuli always evoke an excitatory input on postural muscles, but the stimulus structure produces different postural effects. Peripheral optic flow stimuli stabilize postural sway, while random and foveal optic flow provoke larger sway variability similar to those evoked in the absence of visual stimulation.

Nature provides us with many examples of ingenious sensors and systems at the service of animal behavior, which can be transferred into innovative systems for the control of MicroAir Vehicles (MAVs). Winged insects demonstrate daunting behaviors in spite of the poor resolution of their visual system. For more than 100 million years, they have been navigating in unfamiliar 3D environments by relying on optic flow (OF) cues. To sense and react to the flowing image of the environment, insects are equipped with smart sensors called Elementary Motion Detectors (EMDs) that can act as optic flow sensors (figure 1). The principle of our bio-inspired optic flow sensors is based on findings obtained at our laboratory on the common housefly’s EMDs, by performing electrophysiological recordings on single neuron while applying optical microstimuli to two single photoreceptors cells within a single ommatidium (Franceschini, 1985, Franceschini et al. 1989). The OF-field gives the angular velocity (in rad/s) at which any contrasting object in the environment is moving past the eye (Koenderink, 1986). One lesson we have learned from insects is that they are able to navigate swiftly through the most unpredictable environments without using any velocimeters or rangefinders. Insects rely on optic flow to avoid collisions (Collett, 1980; Wagner, 1982; Tammero and Dickinson, 2002), to follow a corridor (Kirchner and Srinivasan, 1989; Baird et al. 2005; Ruffier et al., 2007; Serres et al., 2007; Serres et al. 2008), to follow the terrain (William, 1965; Srygley and Oliveira, 2001), to fly against wind (Kennedy 1939, Kennedy 1951), and to cruise and land (Srinivasan et al., 1996), for example. Interestingly, insects seem to maintain a constant optic flow with respect to their surrounding environment while cruising and landing (Kennedy, 1951; David, 1978, Srinivasan et al. 1996, 2000). Several MAV autopilots were built in recent years which show how insects could achieve this feat by using a feedback control system called the optic flow regulator (Ruffier and Franceschini, 2003, 2005, Serres et al. 2008). Future MAVs’ visual guidance systems may have to incorporate optic flow sensors covering various parts of the visual field, like insects do (Figure 2). The biorobotic approach developed at our laboratory over the past 20 years enabled us to construct several terrestrial and aerial robots based on OF sensing (Pichon et al., 1989, Franceschini et al., 1992, 1997; Mura and Franceschini, 1996; Netter and Franceschini., 2002,