A computational theory of visual receptive fields

Visual neuroscientists seek to answer two related questions. First, what does the visual system do? Second, how does it do it? While an answer to the second question is a description based on anatomy and biophysics, an answer to the first question is a description of computations performed on images. Important steps in providing this description were made with the publication of two classical studies on primary visual cortex by Movshon et al. (1978a,b). These studies have not aged in their readability, and do not require an interpretive key to be enjoyed by the contemporary reader. As an introduction, however, this Perspectives article briefly reviews the state of the art at the time of their appearance, their main findings, and their influence on three decades of subsequent investigations. The 1960s brought important advances in the understanding of the computations performed by the retina. An influential view emerged that described retinal ganglion cells as linear filters, i.e. as processes that compute a weighted sum of the intensities in the stimulus, with weights given by the receptive field (Enroth-Cugell & Robson, 1966). This simple 'linear model' applied to the majority of ganglion cells in the cat retina (those of the X type, corresponding to P cells in primates). It was quite powerful, as it promised to predict the responses of X ganglion cells to arbitrary stimuli, on the basis of the simple knowledge of the receptive field. This purely linear view was complemented by the discovery that another kind of ganglion cells, those of Y type (which correspond to M cells in primates), perform non-linear operations, suggesting that they sum the distorted output of subunits that in turn have linear receptive fields (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976). This arrangement confers position invariance: for stimuli of high spatial frequency, Y cells respond equally regardless of position within the receptive field. These advances suggested new ways of looking at simple and complex cells in primary visual cortex. Hubel & Wiesel (1962) had defined simple cells as having distinct antagonistic regions in their receptive fields, and had suggested that knowing those regions, one could predict 'the responses to any shape of stimulus, stationary or moving'. They had defined complex cells as any cell that was not simple, and had reported that complex cells achieved position invariance within their receptive field: they would respond to a stimulus of the appropriate orientation regardless of position within the receptive field. These attributes of simple and complex cells resembled those of X and Y ganglion cells. Some authors proposed that the correspondence could be anatomical, i.e. that it reflected predominance of X inputs to simple cells and of Y inputs to complex cells (Stone, 1972; Movshon, 1975), a suggestion that was not later confirmed. More generally, however, the linear and non-linear models and the stimulation procedures that had been so useful to analyse X and Y cells (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976) constituted promising starting points for a concise and precise characterization of simple and complex cells. In addition to the breakthroughs in retinal physiology, another force was pushing towards the use of quantitative engineering techniques in primary visual cortex: such techniques were proving successful to study human perception. In particular, the University of Cambridge – where Movshon, Thompson and Tolhurst operated – was a hotbed of research into the relations between single neuron responses and perceptual phenomena. These phenomena were investigated with rigorous psychophysical measures, and described with quantitative models based on image filtering (Graham, 1989; Wandell, 1995). Much of this research rested on the concept of 'channels', which are linear filters. Simple cells in the cortex seemed to be good candidates for such a role. Did they exhibit linear summation? The time had come to apply the power of linear systems analysis and related techniques to primary visual cortex. In their first article, Movshon et al. (1978a) applied linear systems analysis to the responses of simple cells. They measured responses of simple cells to gratings and bars, and asked if such responses were consistent with the output of a linear receptive field (Fig. 1A). The models of simple and complex cells proposed by Movshon, Thompson and Tolhurst ( Movshon et al. 1978a,b) A, linear model of simple cells. The first stage is linear filtering, i. e. aweighted sum of the image intensities, with weights given by the receptive field. The second stage isrectification: only the part of the responses that is larger than a threshold is seen in the firing rate response. B, subunit model of complex cells. The first stage is linear filtering by a number of receptive fields such asthose of simple cells (here we show four of them with spatial phases offset by 90 deg). The subsequent stages involve rectification, and then summation. Much as Enroth-Cugell & Robson (1966) had done for X retinal cells, they asked if the responses to drifting sinusoidal stimuli were sinusoidal, as one would expect from a linear filter. The results supported this view, provided that the responses were rectified by the spike threshold, which shows only the part of the responses that lie above threshold (Fig. 1A). The authors then asked how the responses depended on the spatial phase of a standing grating whose contrast oscillated sinusoidally in time. This test had been applied to X and Y retinal cells by Hochstein & Shapley (1976). Here the linear model was put to a quantitative test, and the fit was good, provided again that the output of the receptive field was passed through a rectification stage that thresholded it (Fig. 1A). Movshon, Thompson and Tolhurst were even able to suggest how high the threshold should be relative to rest. They expressed this threshold in the units of firing rate responses, spikes s−1. For example, for the cell in their Fig. 4, the estimated threshold was 8 spikes s−1 (if the receptive field were to output 12 spikes s−1, the neuron would output 4 spikes s−1). Finally, the authors asked a key question: was the selectivity of simple cells predictable on the basis of the receptive field alone, as had been suggested by Hubel and Wiesel, and as would be expected of a linear filter? To test this hypothesis, they turned again to the approach that had demonstrated the linearity of X retinal cells (Enroth-Cugell & Robson, 1966). First, they measured responses to drifting gratings of various spatial frequencies (Fig. 2A). Then, they measured responses to bars flashed at various positions, thus estimating the profile of the receptive field (Fig. 2B, histograms). According to the linear hypothesis, the first data set could be used to predict the second one. This was indeed the case for the cells (Fig. 2B, curve): just as predicted by the linear model, the selectivity of simple cells for spatial frequency could be predicted on the basis of the receptive field profile. Linearity of spatial summation in simple cells the experiment in Fig. 9 of the first 1978 article by Movshon, Thompson and Tolhurst ( Movshon et al. 1978a) Responses were simulated from a model simple cell with a linear spatial receptive field that summates strongly distorted thalamic inputs (Carandini et al. 2002). A, spatial frequency tuning of the simple cell. The ordinate marks the amplitude of the sinusoidal modulation caused by drifting sinusoidal gratings, whose spatial frequency is plotted on the abscissa. B, profile of the receptive field of the simple cell. The histogram shows the firing rate elicited by flashing bars in various spatial positions across the receptive field. Negative responses indicate responses elicited when withdrawing the bar. The curve shows the prediction based on linearity, obtained by Fourier transform of the data in A. For all its success in explaining responses of simple cells, the linear model (Fig. 1A) could not possibly work for complex cells. Complex cells are insensitive to the precise position of a bar within the receptive field, and respond both to the onset and to the offset of the bar. Neither of these properties could arise from a single linear receptive field. Hubel & Wiesel (1962) had therefore described complex cells as summing the output of a number of simple cells with similar orientation preference but different receptive field profiles. Borrowing from the subunit model proposed for Y ganglion cells by Hochstein & Shapley (1976), such a description could be made quantitative by postulating a number of linear receptive fields orientated in space (subunits), whose outputs are rectified by threshold, and integrated into a single response (Fig. 1B). In their second article, Movshon et al. (1978b) went on to propose such a subunit model for complex cells (Fig. 1B), and to justify each of its components. They started by performing the same three measures that they had performed in simple cells. First, the authors studied the modulation in firing rate caused by drifting sinusoidal gratings. Complex cells responded phasically only at the lowest spatial frequencies, but as soon as the frequency approached the optimal, the responses became constant in time. This was consistent with a subunit model (Fig. 1B). Each of the subunits would respond with a sinusoid, but the rectified sinusoids would be offset in time, and therefore they would sum to an approximately constant value. Second, they asked how complex cell responses depended on the spatial phase of a standing grating whose contrast reversed in time. Just as for Y ganglion cells (Hochstein & Shapley, 1976), the cells did not care for spatial phase, giving two responses for each cycle of the stimulus (once for each sign of contrast). These results again were consistent with the subunit model (Fig. 1B): each subunit would give positive responses only once in each cycle, and only at specific spatial phases, but the sum of the positive subunit responses would rise twice in each cycle, irrespective of spatial phase. Third, they asked whether the selectivity of complex cells was predictable on the basis of the receptive field alone. The answer was a resounding no: as predicted by the subunit model (Fig. 1B), receptive fields of complex cells don't have strong distinct subregions, so it is not possible to predict their selectivity from a receptive field profile. While all these results seemed consistent with a subunit model (Fig. 1B), a key attribute of that model remained to be verified: are the receptive fields of the subunits really linear? To test this hypothesis, the authors devised an elegant experiment, in which they measured the interaction of dark and light bars in different positions of the receptive field. To understand this experiment, consider a simplified model of complex cell that includes only four subunits (Fig. 1B), and imagine placing a bright bar over the central positive region in the receptive field of the first subunit. This subunit will give a strong positive response, and will thus operate well above threshold. Conversely, the third subunit, which has the opposite receptive field, will be strongly suppressed, and will operate much below threshold. The second and fourth subunits, in turn, will barely respond, so they will operate (as normally) slightly below threshold. Adding a second bar to the stimulus therefore will mostly reveal the receptive field properties of the first subunit. Indeed, as shown in Fig. 8 of the original article, the interaction profiles between bars resemble receptive fields. If these receptive fields of the subunits operate linearly (and if all subunits are similar in spatial frequency preferences), then it should be possible to predict the spatial frequency selectivity of the neuron based on the receptive field profile of the subunits, just as for simple cells (Fig. 2). The results of this experiment, illustrated in Fig. 9 of the original article, confirm this prediction, providing strong support for the subunit model of complex cells (Fig. 1B). The results of the two articles therefore can be summarized by two models: the linear model of simple cells (Fig. 1A), and the subunit model of complex cells (Fig. 1B). These models share two key attributes: (1) all the image processing is performed by linear filters; (2) the non-linearities operate on the time-varying signals that are output by the filters. These models have formed the basis for much that has followed in the subsequent three decades. There is obviously no space here to cover this territory, for which we refer the reader to recent reviews (e.g. Carandini et al. 2005). What might be more useful would be to discuss those aspects that, with 100% hindsight, could have been analysed differently, and would arguably have led to slightly different conclusions. First, when comparing responses to gratings to the profiles of the receptive field (Fig. 2), whether for simple cells or for the subunits of complex cells, a free scaling factor was allowed to obtain a match. Such a scaling factor should not be necessary for the simple models shown in Fig. 1, but it is necessary for actual cells, whose responsiveness depends very much on the local distribution of contrast. To account for this dependence, the models were later extended to include a divisive stage that controls responsiveness on the basis of the distribution of local contrast (for reviews see Heeger, 1992; Carandini et al. 1999). Second, the authors were perhaps wise to concentrate on one spatial dimension, and thus to avoid the contentious issue of orientation selectivity. Orientation tuning curves were not measured in this study, arguably because rotating the stimulation device involved placing one's hand within centimetres of electrocution, something that was done only reluctantly, once for each cell (J. A. Movshon, personal communication). Had the authors measured 2-dimensional receptive field profiles they could have asked whether these profiles predicted orientation selectivity. This issue remained open for decades and is not entirely closed to this date (reviewed by Ferster & Miller, 2000). Finally, perhaps the greatest limitation of these studies is that they concentrated on the spatial domain, and did not test linearity in the temporal domain. On the one hand, as was shown in the subsequent decades, the concept of spatial receptive field can be fruitfully extended to 3-dimensional space–time, to account for phenomena of direction selectivity (in fact, the nascent signs of such an extension can be seen already in the second 1978 article). On the other hand, as demonstrated later by Tolhurst et al. (1980), in primary visual cortex temporal summation is far from linear: responses are much more transient than would be expected from the frequency selectivity curves. This non-linearity is puzzling: how could a receptive field be spatially linear and temporally non-linear? It is now thought that because a cortical cell sums inputs from a variety of spatially displaced thalamic neurons, even if the individual inputs are grossly distorted by saturating and threshold non-linearities, the overall spatial summation properties of the neuron will remain approximately linear (Carandini et al. 2002; Priebe & Ferster, 2006). Indeed, the simulations shown here (Fig. 2) resulted from a model cell that summates such strongly non-linear inputs. This model cell passes the test of spatial linearity devised by the 1978 papers (Fig. 2), yet it would fail any test of temporal linearity. If it took three decades to obtain this realization, it is possibly because these 1978 papers made such a compelling case for linearity, and the field took them as evidence that every step in the visual system up to the primary visual cortex had to be linear. The reasoning, partly explicit in these papers and implicit in much of the subsequent literature, was that there could be no non-linear stages from the cones to the cortex because those stages would have prevented the cortical cells from passing the linearity tests. These are modest limitations, and they are evident only with three decades of hindsight. Overall the impact of these papers on the field was forceful and positive. As is evident from glancing at any recent review (e.g. Carandini et al. 2005) much of what was done to this date to explain responses of primary visual cortex, e.g. to explain properties such as orientation selectivity, direction selectivity and binocular integration, rested on the results of these two classical papers. Moreover, these studies succeeded in providing a foundation for the models of pattern perception based on psychophysical channels (Graham, 1989). Our only hope is that similarly powerful quantitative studies will soon appear for areas beyond the primary visual cortex.

This paper presents an in-depth theoretical analysis of the orientation selectivity properties of simple cells and complex cells, that can be well modelled by the generalized Gaussian derivative model for visual receptive fields, with the purely spatial component of the receptive fields determined by oriented affine Gaussian derivatives for different orders of spatial differentiation. A detailed mathematical analysis is presented for the three different cases of either: (i) purely spatial receptive fields, (ii) space-time separable spatio-temporal receptive fields and (iii) velocity-adapted spatio-temporal receptive fields. Closed-form theoretical expressions for the orientation selectivity curves for idealized models of simple and complex cells are derived for all these main cases, and it is shown that the orientation selectivity of the receptive fields becomes more narrow, as a scale parameter ratio κ, defined as the ratio between the scale parameters in the directions perpendicular to vs. parallel with the preferred orientation of the receptive field, increases. It is also shown that the orientation selectivity becomes more narrow with increasing order of spatial differentiation in the underlying affine Gaussian derivative operators over the spatial domain. A corresponding theoretical orientation selectivity analysis is also presented for purely spatial receptive fields according to an affine Gabor model, showing that: (i) the orientation selectivity becomes more narrow when making the receptive fields wider in the direction perpendicular to the preferred orientation of the receptive field; while (ii) an additional degree of freedom in the affine Gabor model does, however, also strongly affect the orientation selectivity properties.

Event Abstract Back to Event Eyes are not cameras: Deriving properties of retinal ganglion cells from the natural input Daniela Filipa G. Pamplona1, Jochen Triesch1 and Constantin A. Rothkopf1* 1 Frankfurt Institute for Advanced Studies (FIAS), Germany The shape of retinal ganglion cells has been related to the statistics of the natural environment under the efficient coding hypothesis. The statistical regularities present in natural images have been measured and neurons have been shown to reduce the redundancy present in these stimuli. This analysis reveals that retinal Ganglion cells’ properties can be related to the second order dependencies present in natural images. While such analysis has used the convenient assumption that photographic pictures of natural environments are equivalent to the natural input to the visual system, we here consider quantitatively the influence of the imaging process of a model eye on these statistics. This analysis reveals strong dependencies of the local intensity statistics on the position across the visual field thereby explaining substantial anisotropies of retinal Ganglion cells reported in the literature. First, we generated artificial scenes with three-dimensional edge elements and quantified the resulting distributions of orientations by applying the perspective projection onto a spherical surface. These distributions show a strong influence of the imaging process on the statistics of the input to the visual system. Secondly, image data from a naturalistic virtual environment was obtained. The second order statistics were computed as a function of eccentricity and radial distance from the center of projection. This analysis confirms strong dependencies of the second order statistics on the position across the visual field. Finally, we repeated the analysis to commonly used natural image databases including the van Hateren database and quantified the second order dependencies as function of the position across the visual field using a new generalized parameterization of the power spectra. We conclude by providing a detailed quantitative analysis of the second order statistical dependencies of the natural input to the visual system and making novel predictions about retinal Ganglion cells’ receptive field profiles as function of their position. Acknowledgements Supported by the BMBF through the Bernstein Focus: Neurotechnology Frankfurt. Keywords: efficient coding, natural image statistics, power spectrum, projective geometry, receptive field, retinal ganglion cell Conference: Bernstein Conference 2012, Munich, Germany, 12 Sep - 14 Sep, 2012. Presentation Type: Poster Topic: Sensory processing and perception Citation: Pamplona DG, Triesch J and Rothkopf CA (2012). Eyes are not cameras: Deriving properties of retinal ganglion cells from the natural input. Front. Comput. Neurosci. Conference Abstract: Bernstein Conference 2012. doi: 10.3389/conf.fncom.2012.55.00074 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 21 May 2012; Published Online: 12 Sep 2012. * Correspondence: Dr. Constantin A Rothkopf, Frankfurt Institute for Advanced Studies (FIAS), Frankfurt am Main, D-60438, Germany, constantin.rothkopf@cogsci.tu-darmstadt.de Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Daniela Filipa G Pamplona Jochen Triesch Constantin A Rothkopf Google Daniela Filipa G Pamplona Jochen Triesch Constantin A Rothkopf Google Scholar Daniela Filipa G Pamplona Jochen Triesch Constantin A Rothkopf PubMed Daniela Filipa G Pamplona Jochen Triesch Constantin A Rothkopf Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Established theoretical reasons recommend functions that are limited in space and spatial frequency as models for receptive fields of visual neurons. On the basis of an interpretation of spread as variance and via analogy to quantum mechanics, the Gabor functions are often stated to be maximally limited in space and spatial frequency (Daugman, 1985). Some Gabor functions indeed resemble receptive fields in V1, but this view provides little insight into why V1 receptive fields have only a small number of lobes, nor into receptive field shape beyond V1. Here we consider an alternative interpretation of “limited in space and spatial frequency.” We consider a function to be “confined” in space (or spatial frequency) if it is unchanged by windowing in space (or spatial frequency). While no function can be simultaneously confined in both space and spatial frequency, there is a rigorous sense in which the 2-dimensional Hermite functions achieve simultaneous confinement as nearly as possible. The 2-dimensional Hermite functions are a complete basis set and form a natural hierarchy. The first levels of this hierarchy contain functions that resemble Gabor functions with a small number of lobes, and thus resemble V1 receptive fields. Further down the hierarchy are intrinsically 2-dimensional functions, some of which resemble the non-Cartesian gratings, to which some V4 neurons respond preferentially (Gallant, 1996). In addition to their many interesting mathematical properties, the two-dimensional Hermite functions allow for efficient (“sparse”) local synthesis of images, including natural scenes, faces, and letters. While we make no claim that this view suffices to account for receptive field structure, we suggest that it provides a framework for a principled study of receptive fields, and that it is useful to think of receptive fields (along the V1-to-V2-to-V4 pathway) as not only expanding, but also increasing in their combined space-bandwidth aperture.

This work presents a first evaluation of using spatio-temporal receptive fields from a recently proposed time-causal spatio-temporal scale-space framework as primitives for video analysis. We propose a new family of video descriptors based on regional statistics of spatio-temporal receptive field responses and evaluate this approach on the problem of dynamic texture recognition. Our approach generalises a previously used method, based on joint histograms of receptive field responses, from the spatial to the spatio-temporal domain and from object recognition to dynamic texture recognition. The time-recursive formulation enables computationally efficient time-causal recognition. The experimental evaluation demonstrates competitive performance compared to state of the art. In particular, it is shown that binary versions of our dynamic texture descriptors achieve improved performance compared to a large range of similar methods using different primitives either handcrafted or learned from data. Further, our qualitative and quantitative investigation into parameter choices and the use of different sets of receptive fields highlights the robustness and flexibility of our approach. Together, these results support the descriptive power of this family of time-causal spatio-temporal receptive fields, validate our approach for dynamic texture recognition and point towards the possibility of designing a range of video analysis methods based on these new time-causal spatio-temporal primitives.

[This corrects the article DOI: 10.3389/fnins.2019.00979.].

The functional properties of neural sensory cells or small neural ensembles are often characterized by analyzing response-conditioned stimulus ensembles. Many widely used analytical methods, like receptive fields (RF), Wiener kernels or spatio-temporal receptive fields (STRF), rely on simple statistics of those ensembles. They also tend to rely on simple noise models for the residuals of the conditional ensembles. However, in many cases the response-conditioned stimulus set has more complex structure. If not taken explicitly into account, it can bias the estimates of many simple statistics, and lead to erroneous conclusions about the functionality of a neural sensory system. In this article, we consider sensory noise in the visual system generated by small stimulus shifts in two dimensions (2 spatial or 1-space 1-time jitter). We model this noise as the action of a set of translations onto the stimulus that leave the response invariant. The analysis demonstrates that the spike-triggered average is a biased estimator of the model mean, and provides a de-biasing method. We apply this approach to observations from the stimulus/response characteristics of cells in the cat visual cortex and provide improved estimates of the structure of visual receptive fields. In several cases the new estimates differ substantially from the classic receptive fields, to a degree that may require re-evaluation of the functional description of the associated cells.

To form a reliable, consistent, and accurate representation of the acoustic scene, a reasonable conjecture is that cortical neurons maintain stable receptive fields after an early period of developmental plasticity. However, recent studies suggest that cortical neurons can be modified throughout adulthood and may change their response properties quite rapidly to reflect changing behavioral salience of certain sensory features. Because claims of adaptive receptive field plasticity could be confounded by intrinsic, labile properties of receptive fields themselves, we sought to gauge spontaneous changes in the responses of auditory cortical neurons. In the present study, we examined changes in a series of spectrotemporal receptive fields (STRFs) gathered from single neurons in successive recordings obtained over time scales of 30-120 min in primary auditory cortex (A1) in the quiescent, awake ferret. We used a global analysis of STRF shape based on a large database of A1 receptive fields. By clustering this STRF space in a data-driven manner, STRF sequences could be classified as stable or labile. We found that >73% of A1 neurons exhibited stable receptive field attributes over these time scales. In addition, we found that the extent of intrinsic variation in STRFs during the quiescent state was insignificant compared with behaviorally induced STRF changes observed during performance of spectral auditory tasks. Our results confirm that task-related changes induced by attentional focus on specific acoustic features were indeed confined to behaviorally salient acoustic cues and could be convincingly attributed to learning-induced plasticity when compared with "spontaneous" receptive field variability.

In our previous study of the cytoarchitectonic field 7 of cat cortex we had described neurons with extremely elongated receptive fields (RFs). The long axes of these RFs were oriented radially, towards the centre of the retina. These neurons represented only the lower contralateral part of visual field. They were surrounded from all sides by neurons with clearly different RF properties. We proposed that neurons with a similar radial organization and with RFs in the upper visual field also exist in the cortex but are localized in the area that was distant from the representation of the corresponding lower visual field. We expected to find these neurons in front of the representation of the upper visual field in areas V1, V2 and V3 (fields 17, 18 and 19), behind the central representation in area 21a. This cortical region was studied in five behaving cats. In all animals, neurons with radial RFs in the upper visual field were found in the expected location. As in the lower visual field, their RFs always spared the central visual field. Other RF properties of these neurons were also very similar to those found previously in the lower visual field. It became obvious that neurons with radial RFs are included into the fourth extrastriate crescent with complete contralateral representation. However, in the fourth crescent, RF properties in the central visual field differed significantly from those on the periphery. As a result, neurons with similar radial RFs in the upper and lower visual fields were located in the distant cortical regions, and were separated by the representation of the central visual field presented by the non-radial neurons of the cytoarchitectonic area 21a.

The spectro-temporal receptive field (STRF) is a model representation of the excitatory and inhibitory integration area of auditory neurons. Recently it has been used to study spectral and temporal aspects of monaural integration in auditory centers. Here we report the properties of monaural STRFs and the relationship between ipsi- and contralateral inputs to neurons of the central nucleus of cat inferior colliculus (ICC) of cats. First, we use an optimal singular-value decomposition method to approximate auditory STRFs as a sum of time-frequency separable Gabor functions. This procedure extracts nine physiologically meaningful parameters. The STRFs of approximately 60% of collicular neurons are well described by a time-frequency separable Gabor STRF model, whereas the remaining neurons exhibited obliquely oriented or multiple excitatory/inhibitory subfields that require a nonseparable Gabor fitting procedure. Parametric analysis reveals distinct spectro-temporal tradeoffs in receptive field size and modulation filtering resolution. Comparisons between an identical model used to study spatio-temporal integration areas of visual neurons further shows that auditory and visual STRFs share numerous structural properties. We then use the Gabor STRF model to compare quantitatively receptive field properties of contra- and ipsilateral inputs to the ICC. We show that most interaural STRF parameters are highly correlated bilaterally. However, the spectral and temporal phases of ipsi- and contralateral STRFs often differ significantly. This suggests that activity originating from each ear share various spectro-temporal response properties such as their temporal delay, bandwidth, and center frequency but have shifted or interleaved patterns of excitation and inhibition. These differences in converging monaural receptive fields expand binaural processing capacity beyond interaural time and intensity aspects and may enable colliculus neurons to detect disparities in the spectro-temporal composition of the binaural input.

This article gives an overview of a normative theory of visual receptive fields. We describe how idealized functional models of early spatial, spatio-chromatic and spatio-temporal receptive fields can be derived in a principled way, based on a set of axioms that reflect structural properties of the environment in combination with assumptions about the internal structure of a vision system to guarantee consistent handling of image representations over multiple spatial and temporal scales. Interestingly, this theory leads to predictions about visual receptive field shapes with qualitatively very good similarities to biological receptive fields measured in the retina, the LGN and the primary visual cortex (V1) of mammals.

We present a spike-triggered averaging method capable of mapping the visual receptive fields of several neurons simultaneously. The stimulation is general and the mapping proceeds automatically without the need to match the stimulation to the cells' preference for position, orientation, direction, etc. The maps are spatiotemporal; receptive field (RF) structures are quantitatively determined in three dimensions: the two dimensions of visuotopic space, and time. The method presented is one of a family of "reverse correlation" or "spike-triggered averaging" techniques (DeBoer and Kuyper 1968) capable of revealing linear aspects of stimulus-response coupling. The formal relationship of these methods to stimulus-response cross-correlation is shown. The analysis is extended to provide some second-order axis-of-motion information ("direction marks"). The stimulus is a constantly illuminated, randomly jumping bright or dark spot, not an elongated bar. Spot diameters between one-third to 1 x RF width are effective. The method ascertains for each recorded action potential or "spike" the prior visual field position of the spot. The average or most probable spot positions define the receptive field spatially. Repeating the process for a succession of times prior to observed spikes defines the field temporally, presented here as a succession of spatial maps. We term this portrayal a receptive field cinematogram, RFc or ciné. The RFc reveals and economically portrays the spread of excitability and suppression across the receptive field, culminating in the generation of a spike. RFcs for LGN neurons and for simple cells recorded in cat cortical areas 17 and 18 are presented and interpreted in terms of classic ON/OFF regions. The availability of temporal information permits the separation of an excitatory exit response, generated when a moving bright spot leaves an OFF region, from an excitatory entrance response occurring when a bright spot enters an ON region, because these responses occur at different times (exit responses earlier). Spike emission remains coupled to (cross-correlated with) stimulus events over time periods as long as 96 ms, implying that some stimulus drive or afferent visual input is delayed by as much as 96 ms more than other input. This is a striking instance of temporal dispersion in the visual system. In some cells, said to be "spatiotemporally inseparable", the delay (latency) varies systematically across the visual field; i.e., the place for optimal stimulation varies with the time prior to spike emission. In these cells, the RFc shows receptive field structures which move across the visual field over trajectories equal to approximately twice the total conventional RF width.(ABSTRACT TRUNCATED AT 400 WORDS)

The visual system faces a trade-off between increased spatial integration of disparate local signals and improved spatial resolution to filter out irrelevant noise. Increased spatial integration is beneficial when signals are weak, whereas increased spatial resolution is particularly beneficial when focusing on a small object in a cluttered natural scene. The receptive field (RF) size of visual cortical neurons can be modulated depending on various factors such as sensory context, allowing adaptive integration of sensory signals. In this study, we explored the spatial integration properties of neurons in macaque middle temporal visual area (MT). We hypothesized that spatial resolution would increase when high-contrast noise was presented simultaneously with a visual stimulus, enabling focus on a small object in a cluttered scene. To test this hypothesis, we mapped the RFs of MT neurons of two fixating monkeys in a 5 × 5 grid manner using a small patch of random-dot motion. To examine the effects of noise on RF profile, a dynamic noise (0% coherence dots) of varying diameter was concurrently presented at the RF center. We found that RF size decreased when noise diameter increased. Analyses based on the response normalization model and area summation provided evidence for the potential contribution of spatial summation properties within the RF and surround suppression to the apparent contraction of RF size. Our results suggest that MT neurons integrate smaller regions of motion signals when signals are embedded in noise, an efficient strategy to filter out surrounding noise.

Contribution of linear spatiotemporal receptive field structure to velocity selectivity of simple cells in area 17 of cat

One of the long-term goals of sensory neuroscience is the development of sound experimental and theoretical methods for understanding the functional organization of sensory systems. In this regard, Drosophila melanogaster is the model organism of choice: it boasts a relatively small brain, its sensory systems have been anatomically well characterized and it offers an extensive genetic toolbox for visualizing and altering its neural circuits. Despite these advantages however, comprehensive models of sensory processing in Drosophila are sparse, in particular due to the lack of methods for estimating spike-processing neural circuits in higher brain centers. The majority of existing neural circuit models and methods for their identification assume rate-based systems (see [1] for a review), and take both the input (stimuli) and the output (response rates) to be in the continuous domain. In a practical setting, however, outputs of most neurons in a sensory system are sequences of all-or-none action potentials. Furthermore, input signals are continuous only for those neurons that are located at the sensory periphery. In contrast, input signals for neurons upstream of sensory neurons are spatiotemporal spike trains. Hence, there is a need to develop a generic framework for estimation of both receptive fields in the periphery and of spatiotemporal spike processing upstream. Here we propose a novel theoretical approach for estimating receptive fields in circuit models that incorporate biophysical spike-generating mechanisms (e.g., the Hodgkin-Huxley neuron) and admit both continuous sensory signals and multidimensional spike trains as input stimuli. We thus explicitly take into account the highly nonlinear nature of spike generation that has been shown to result in significant interactions between various stimulus features [2], [3] and to fundamentally affect the estimation of receptive fields [4]. Furthermore, and in contrast to many existing methods [1], our approach estimates receptive fields directly from spike times produced by a neuron, thereby obviating the need to repeat experiments in order to compute the neuron’s instantaneous rate of response (e.g., PSTH). The employed test signals belong to spaces of bandlimited functions and bridge the gap between identification using synthetic and naturalistic stimuli. This makes our methodology particularly attractive in those sensory modalities (most notably olfaction [5]), where it is difficult to produce stimuli that are white and/or have particular distribution/ attributes [1]-[4]. First, we work out in detail algorithms for identifying temporal, spatial and spatiotemporal receptive fields in the sensory periphery. We show that our methodology is readily generalizable to multiple receptive fields as well as to higher dimensions, allowing one to consider more complex receptive fields, if needed. Second, we demonstrate how to identify the processing of multiple spiking inputs converging onto the dendritic tree of a spiking neuron. Third, we show that the presented methodology allows one to model integration of sensory modalities in higher brain centers. Finally, we test the proposed approach using in-vivo data recorded from the olfactory system of Drosophila.