Beyond ℓ1 sparse coding in V1

Retinal ganglion cells in the turtle were extracellularly recorded to define the shapes of their receptive fields by small moving light spots. To better define the geometries, spectral-light adaptations and vitreal injections of 2-amino-4-phosphonobutyric acid (APB) were used to disrupt balances in field organization along dimensions of wavelength, ON and OFF responses, and center/surround areas. Three-dimensional data plots were fit by Gaussian, Gabor, and cardioid functions to show that the shapes of receptive fields are predicted by combinations of these multiplied functions. Results indicate that Gaussian functions describe simple symmetrical receptive fields that are center-only; Gabor functions describe center/surround color-opponent receptive fields that have a ring of spike activity in the periphery; and directionally selective receptive fields, in contrast, which are asymmetrical, are described by cardioid functions adjoined to Gaussian or Gabor functions. The advantage of linking multiplied functions is that receptive fields are unified by a model that predicts progressively more complex field geometries derived from particular stimulating conditions.

A receptive field constitutes a region in the visual field where a visual cell or a visual operator responds to visual stimuli. This paper presents a theory for what types of receptive field profiles can be regarded as natural for an idealized vision system, given a set of structural requirements on the first stages of visual processing that reflect symmetry properties of the surrounding world. These symmetry properties include (i) covariance properties under scale changes, affine image deformations, and Galilean transformations of space–time as occur for real-world image data as well as specific requirements of (ii) temporal causality implying that the future cannot be accessed and (iii) a time-recursive updating mechanism of a limited temporal buffer of the past as is necessary for a genuine real-time system. Fundamental structural requirements are also imposed to ensure (iv) mutual consistency and a proper handling of internal representations at different spatial and temporal scales. It is shown how a set of families of idealized receptive field profiles can be derived by necessity regarding spatial, spatio-chromatic, and spatio-temporal receptive fields in terms of Gaussian kernels, Gaussian derivatives, or closely related operators. Such image filters have been successfully used as a basis for expressing a large number of visual operations in computer vision, regarding feature detection, feature classification, motion estimation, object recognition, spatio-temporal recognition, and shape estimation. Hence, the associated so-called scale-space theory constitutes a both theoretically well-founded and general framework for expressing visual operations. There are very close similarities between receptive field profiles predicted from this scale-space theory and receptive field profiles found by cell recordings in biological vision. Among the family of receptive field profiles derived by necessity from the assumptions, idealized models with very good qualitative agreement are obtained for (i) spatial on-center/off-surround and off-center/on-surround receptive fields in the fovea and the LGN, (ii) simple cells with spatial directional preference in V1, (iii) spatio-chromatic double-opponent neurons in V1, (iv) space–time separable spatio-temporal receptive fields in the LGN and V1, and (v) non-separable space–time tilted receptive fields in V1, all within the same unified theory. In addition, the paper presents a more general framework for relating and interpreting these receptive fields conceptually and possibly predicting new receptive field profiles as well as for pre-wiring covariance under scaling, affine, and Galilean transformations into the representations of visual stimuli. This paper describes the basic structure of the necessity results concerning receptive field profiles regarding the mathematical foundation of the theory and outlines how the proposed theory could be used in further studies and modelling of biological vision. It is also shown how receptive field responses can be interpreted physically, as the superposition of relative variations of surface structure and illumination variations, given a logarithmic brightness scale, and how receptive field measurements will be invariant under multiplicative illumination variations and exposure control mechanisms.

Established theoretical reasons recommend functions that are limited in space and spatial frequency as models for receptive fields of visual neurons. On the basis of an interpretation of spread as variance and via analogy to quantum mechanics, the Gabor functions are often stated to be maximally limited in space and spatial frequency (Daugman, 1985). Some Gabor functions indeed resemble receptive fields in V1, but this view provides little insight into why V1 receptive fields have only a small number of lobes, nor into receptive field shape beyond V1. Here we consider an alternative interpretation of “limited in space and spatial frequency.” We consider a function to be “confined” in space (or spatial frequency) if it is unchanged by windowing in space (or spatial frequency). While no function can be simultaneously confined in both space and spatial frequency, there is a rigorous sense in which the 2-dimensional Hermite functions achieve simultaneous confinement as nearly as possible. The 2-dimensional Hermite functions are a complete basis set and form a natural hierarchy. The first levels of this hierarchy contain functions that resemble Gabor functions with a small number of lobes, and thus resemble V1 receptive fields. Further down the hierarchy are intrinsically 2-dimensional functions, some of which resemble the non-Cartesian gratings, to which some V4 neurons respond preferentially (Gallant, 1996). In addition to their many interesting mathematical properties, the two-dimensional Hermite functions allow for efficient (“sparse”) local synthesis of images, including natural scenes, faces, and letters. While we make no claim that this view suffices to account for receptive field structure, we suggest that it provides a framework for a principled study of receptive fields, and that it is useful to think of receptive fields (along the V1-to-V2-to-V4 pathway) as not only expanding, but also increasing in their combined space-bandwidth aperture.

Event Abstract Back to Event Spike-triggered covariance and synthetic image replay reveal nonlinearities in V1 color processing Gregory Horwitz1, 2* 1 University of Washington, United States 2 Washington National Primate Research Center, United States The processing of color information in cortical area V1 remains poorly understood. The vastness of the stimulus space is a fundamental problem: not every stimulus pattern can be displayed during a standard neurophysiology experiment. In a previous study, we addressed this problem by stimulating V1 neurons in awake, fixating monkeys with random, colorful patterns and used dimensionality-reduction techniques (averaging and principal components analysis) to isolate features of the stimulus patterns to which the neurons responded1. The stimulus tuning of some neurons was well characterized by a single dimension in the stimulus space. For these neurons, we observed structure in the spike-triggered average stimulus and little or no structure in the principal components of the spike-triggered stimuli. Other neurons responded to multiple spike-triggered stimulus features. These features were manifest in the spike-triggered average and in one or more of the principal components. A subpopulation of neurons had a spike-triggered average that indicated sensitivity to a preferred color throughout the receptive field and a first principal component that indicated sensitivity to luminance edges. These results are inconsistent with a linear model of cone signal integration. We investigated the relationship between chromatic and luminance signals in these neurons by fitting a 2-dimensional linear-nonlinear cascade model to the data. The fits suggested that these neurons carry color-opponent signals, the gain of which is enhanced by luminance contrast. In the current study, we tested this prediction directly by performing the spike-triggered averaging and principal components analyses on the data stream as it was being collected. Spike-triggering stimulus features were extracted from these analyses and linearly combined to create a battery of synthetic images. Synthetic images were displayed at the neuron’s receptive field for 200 ms each, and spikes following each presentation were counted. Results of this experiment confirmed the model predictions: a subset of V1 neurons responded modestly to their preferred color in isolation but strongly to their preferred color superimposed on a luminance edge. We conclude that color processing across a subpopulation of V1 neurons is enhanced at luminance edges. This physiological result may be related to interactions between color and luminance observed psychophysically. Acknowledgments We thank J. Gold and C. Hass for assistance with the computer communication code. This work was supported by the McKnight Foundation and NIH grant RR000166.

The effect of visual crowding within a cortical area V4 neuron’s receptive field (RF) was studied in monkeys trained to discriminate letter-like symbols. The primary task required a fixation of a small dot until it was replaced by a 0.25 deg letter. A discriminative push of a R/L button was required. During fixation stimuli were presented within peripheral (1-8 deg ecc) RFs. The response to a stimulus placed in the center of the RF was examined as a function of the distance to one or two additional stimuli simultaneously presented at flanking positions. Single neurons were recorded with standard electrophysiological techniques. A change in response occurred only when the additional stimuli were within the RF. As the separation between stimuli narrowed within the RF one of three scenarios occurred: the response 1) increased or 2) decreased as the separation decreased, or 3) a decrease in response was rapidly reversed as the stimuli began to merge. The first scenario correlated well with the neuron’s size tuning. Neurons that responded better to larger stimuli, generally summated additional stimuli added to the RF. Neurons that decreased response in presence of flanking stimuli, were usually tuned to stimulus sizes smaller than half the RF size. Rapid reversals of response as stimulus positions merged are consistent with the emergence of a new preferred stimulus. For some neurons the monkeys were required to make a discrimination of targets appearing at the RF center along with the flanking stimuli. Typical visual crowding psychophysical performance was obtained. Attending to the RF stimuli resulted in a general increase in neural activation (relative to an attend-away condition) but did not markedly affect the crowding effect. The change in responsiveness for V4 neurons under visual crowding conditions is best explained as a stimulus integration that produces a less (or more) optimal stimulus configuration. Meeting abstract presented at VSS 2012

Gestalt psychologists in the early part of the century challenged psychophysical notions that perceptual phenomena can be understood from a punctate (‘atomistic’) analysis of the elements present in the stimulus. Their ideas also inhibited later attempts to explain vision in terms of single-unit recordings from individual neurons. A rapprochement between Gestalt phenomenology and physiology seemed unlikely when the first ECVP was held in Marburg, Germany, in 1978. Since that time, response properties of neurons have been discovered that invite an interpretation of visual phenomena (including ‘illusions’) in terms of neuronal processing. Indeed, it is now possible to understand some Gestalt phenomena on the basis of known neurophysiological mechanisms. I begin by outlining the great strides that have been made since the advent of microelectrode recording from single neurons. Initially, cells (‘detectors’) selectively responding to the contrast, spatial frequency, wavelength, orientation, movement, and disparity of a stimulus placed in their receptive fields were used to interpret simple perceptual phenomena (eg, Mach bands, Hermann grids, tilt aftereffect, MAE). In recent years, cells at higher levels of the visual system have been discovered that might explain a number of more complex phenomena: the perception of illusory (occluded) contours by end-stopped cells in area V2, the filling-in of artificial scotomata by neurons in V3, colour constancy by ‘perceptive’ neurons in V4, and the perception of coherent motion in dynamic noise patterns by cells in MT. Studies of flow fields and biological motion in area MST have recently been added to account for our perceptions as we move through our environment. Prompted by these findings, a shift from local to global interactions ‘beyond the classical receptive field’ has taken place in our search for the neural substrates of perception. Current research has focused on three kinds of mechanisms: (i) converging feed-forward projections as the basis for new response properties emerging at higher levels, (ii) recruitment of lateral connections to explain filling-in, and (iii) backward propagation from higher to lower levels to account for binding and figure - ground segregation. How such mechanisms compute large-scale surface properties such as brightness, colour, and depth from local features—indeed how they construct the surfaces themselves from complex natural scenes—is only one of the many questions that are under scrutiny today. Future research will have to tackle the all-important question: How does the analysed information come together again? Furthermore, the contributions of eye movements, attention, learning, other sense modalities, and motor actions will have to be taken into consideration before we arrive at a more complete understanding of visual perception.

This chapter reviews the functional anatomical bases of visual perception in the retina, the lateral geniculate nucleus (LGN) in the visual thalamus, the primary visual cortex (area V1, also called the striate cortex, and Brodmann area 17), and the extrastriate visual cortical areas of the dorsal and ventral pathways. The sections dedicated to the retina and LGN review the basic anatomical and laminar organization of these two areas, as well as their retinotopic organization and receptive field structure. We also describe the anatomical and functional differ- ences among the magnocellular, parvocelullar and koniocellular pathways. The section dedicated to area V1 reviews the functional maps in this area (retino- topic map, ocular dominance map, orientation selectivity map), as well as their anatomical relationship to each other. Special attention is given to the modular columnar organization of area V1, and to the various receptive field classes in V1 neurons. The section dedicated to extrastriate cortical visual areas describes the "where" and "what" pathways in the dorsal and ventral visual streams, and their respective physiological functions. The temporal dynamics of neurons throughout the visual pathway are critical to understanding visibility and neural information processing. We discuss the role of lateral inhibition circuits in processing spatiotemporal edges, corners, and in the temporal dynamics of vision. We also discuss the effects of eye movements on visual physiology and percep- tion in early visual areas. Our visual and oculomotor systems must achieve a very delicate balance: insufficient eye movements lead to adaptation and visual fading, whereas excessive motion of the eyes produces blurring and unstable vision during fixation. These issues are very important for neural prosthetics, in which electrodes are stabilized on the substrate.

The organization and properties of the neural receptive fields in the visual cortex is a prominent factor in understanding the mechanism of neural coding. Here, we perform a systematic and standardized analysis of the structure, organization and properties of the receptive fields in the pan-excitatory neuronal population in the mouse primary visual cortex (area V1). We collected and analyzed a large-scale 2-photon volumetric imaging dataset from an 800x800x600 μm^3 volume in V1 of two mice. The visual responses from were recorded during presentation of a locally sparse noise stimulus to efficiently reconstruct the neural receptive fields. The locally sparse noise stimulus consisted of 9 degree black or white spots on a mean luminance gray background presented at ~3Hz. We designed and implemented a computational pipeline to identify responsive trials, ON and OFF subfields and their geometric properties. Using these experimental and computational pipelines, we found that OFF subfields tended to have larger areas than ON subfields, particularly in layer 2/3. On average, OFF subfields were 15.6 degree^2 larger than the ON subfields across all imaging depths. We also found that the average receptive field area decreased as a function of cortical depth. ON and OFF subfields in layer 5 were on average 22% (ON subfields) and 31.5% (OFF subfields) smaller than those in layer 2/3. Additionally, we found that the responsiveness rate to the sparse noise decreased in deeper layers of cortex. Neurons in layer 2/3 were about 4 times more likely to respond to our visual stimulus compared to neurons in layer 5. Our results provide a systematic and standardized survey of the receptive field properties in V1. The dataset and computational pipeline could serve as a valuable resource to the community to further study the visual responses of neurons in V1.

A neural model of border-ownership from kinetic occlusion.

Silent suppressive surrounds and optimal spatial frequencies of single neurons in cat V1

Reponse suppression by extending sine-wave gratings within the receptive fields of neurons in visual cortical area V3A of the macaque monkey

Natural texture of an object is an important cue for recognition. In real conditions, the incidence angle of light on natural textures leads to a complex pattern of micro-shading that modifies 3D rendering of surfaces. Little is known about visual processing of material properties. The present work aims to study the coding of natural textures by the neurons of area V4 of the awake macaque monkey. We used patches of natural textures issued from the CURET database and illuminated with two or three different angles with their corresponding controls (scrambled Fourier phase). We recorded the responses of V4 neurons to stimuli flashed in their receptive fields (RFs) while the macaques performed a simple fixation task. We show that a large majority of V4 neurons responded to texture patches with a strong modulation across stimuli. The analysis of those responses indicate that V4 neurons integrate first and second order parameters in the image (mean luminance, SNR, and energy), which may be used to achieve texture clustering in a multidimensional space. This clustering was comparable to that of a pyramid of Gabor filters and was not affected by illumination angles. Altogether, these results suggest that the V4 neuronal population acts as a set of filters able to classify textures independently of illumination angle. We conclude that area V4 contains mechanisms that are sensitive to the aspect of textured surfaces, even in an environment where illumination changes continuously.

Simple cells in primary visual cortex were famously found to respond to low-level image components such as edges. Sparse coding and independent component analysis (ICA) emerged as the standard computational models for simple cell coding because they linked their receptive fields to the statistics of visual stimuli. However, a salient feature of image statistics, occlusions of image components, is not considered by these models. Here we ask if occlusions have an effect on the predicted shapes of simple cell receptive fields. We use a comparative approach to answer this question and investigate two models for simple cells: a standard linear model and an occlusive model. For both models we simultaneously estimate optimal receptive fields, sparsity and stimulus noise. The two models are identical except for their component superposition assumption. We find the image encoding and receptive fields predicted by the models to differ significantly. While both models predict many Gabor-like fields, the occlusive model predicts a much sparser encoding and high percentages of ‘globular’ receptive fields. This relatively new center-surround type of simple cell response is observed since reverse correlation is used in experimental studies. While high percentages of ‘globular’ fields can be obtained using specific choices of sparsity and overcompleteness in linear sparse coding, no or only low proportions are reported in the vast majority of studies on linear models (including all ICA models). Likewise, for the here investigated linear model and optimal sparsity, only low proportions of ‘globular’ fields are observed. In comparison, the occlusive model robustly infers high proportions and can match the experimentally observed high proportions of ‘globular’ fields well. Our computational study, therefore, suggests that ‘globular’ fields may be evidence for an optimal encoding of visual occlusions in primary visual cortex.

The current prevailing understanding of sensory processing in cortical areas is largely based on the presentation framework proposes that neural activity form presentations of external stimulus. However, with recent advances in technologies of recording neuronal activity in awake behaving animals, it has been shown that the primary visual cortex (V1) is also strongly driven by self-movement, suggesting that the presentation framework of visual cortex function is incomplete. Particularly, a subset of neurons in V1 selectively respond to transient visuomotor mismatches, and it could be explained by the predictive coding framework which postulates V1 compares the predicted and received visual input and sense the difference between the two. Experience of normal visuomotor coupling has been shown to be crucial for the development of visual-guided behavior, therefore it is likely that the prediction of visual input depends on the experience of visuomotor coupling, which shapes the functional development of V1. In the present study, we use a virtual reality system and trained dark-reared mice with either normal or random visuomotor coupling, and recorded the neural activity in layer 2/3 neurons of V1. We show that mismatch responses in excitatory neurons were strongly dependent on visuomotor experience. By recording several different types of layer 2/3 interneurons in V1 and manipulating their activity with designer receptor exclusively activated by designer drugs (DREADDs) or optogenetics tools, we propose a circuit model showing that the mismatch response could be the result of a disinhibition mediated by local somatostatin (SST) interneurons. Mismatch responses in both groups of mice with different visuomotor training conditions merged together after they were transferred to normal rearing environment. These data demonstrate that neurons in layer 2/3 mouse V1 computes a difference between an excitatory motor-related input and an inhibitory visual input, where the balance between the two inputs is finely tuned by visuomotor experience.

Experience-dependent asymmetric shape of hippocampal receptive fields.