Cross-Modal Competition: The Default Computation for Multisensory Processing.

Superior colliculus (SC) neurons have the ability to synthesize information from different sensory modalities, resulting in enhancements (or depressions) of their activity. This physiological capacity is, in turn, closely tied to changes in overt attentive and orientation responses. The present study shows that, in contrast to more altricial species, many deep layer SC neurons in the rhesus monkey are multisensory at birth. Such neurons can respond to stimuli from different sensory modalities, and all convergence patterns seen in the adult are represented. Nevertheless, these neurons cannot yet synthesize their multisensory inputs. Rather, they respond to combinations of cross-modal stimuli much like they respond to their individual modality-specific components. This immature property of multisensory neurons is in contrast to many of the surprisingly sophisticated modality-specific response properties of these neurons and of their modality-specific neighbors. Thus, although deep SC neurons in the newborn have longer latencies and larger receptive fields than their adult counterparts, they are already highly active and are distributed in the typical adult admixture of visual, auditory, somatosensory, and multisensory neurons. Furthermore, the receptive fields of these neurons are already ordered into well organized topographic maps, and the different receptive fields of the same multisensory neurons show a good degree of cross-modal spatial register. These data, coupled with those from cat, suggest that the capacity to synthesize multisensory information does not simply appear in SC neurons at a prescribed maturational stage but rather develops only after substantial experience with cross-modal cues.

It had previously been shown that influences from two cortical areas, the anterior ectosylvian sulcus (AES) and the rostral lateral suprasylvian sulcus (rLS), play critical roles in rendering superior colliculus (SC) neurons capable of synthesizing their cross-modal inputs. The present studies examined the consequences of selectively eliminating these cortical influences on SC-mediated orientation responses to cross-modal stimuli. Cats were trained to orient to a low-intensity modality-specific cue (visual) in the presence or absence of a neutral cue from another modality (auditory). The visual target could appear at various locations within 45 degrees of the midline, and the stimulus effectiveness was varied to yield an average of correct orientation responses of approximately 45%. Response enhancement and depression were observed when the auditory cue was coupled with the target stimulus: A substantially enhanced probability in correct responses was evident when the cross-modal stimuli were spatially coincident, and a substantially decreased response probability was obtained when the stimuli were spatially disparate. Cryogenic blockade of either AES or rLS disrupted these behavioral effects, thereby eliminating the enhanced performance in response to spatially coincident cross-modal cues and degrading the depressed performance in response to spatially disparate cross-modal cues. These disruptive effects on targets contralateral to the deactivated cortex were restricted to multisensory interactive processes. Orientation to modality-specific targets was unchanged. Furthermore, the pattern of orientation errors was unaffected by cortical deactivation. These data bear striking similarities to the effects of AES and rLS deactivation on multisensory integration at the level of individual SC neurons. Presumably, eliminating the critical influences from AES or rLS cortex disrupts SC multisensory synthesis that, in turn, disables SC-mediated multisensory orientation behaviors.

The present studies were initiated to explore the basis for the response suppression that occurs in cat superior colliculus (SC) neurons when two spatially disparate stimuli are presented simultaneously or in close temporal proximity to one another. Of specific interest was examining the possibility that suppressive regions border the receptive fields (RFs) of unimodal and multisensory SC neurons and, when activated, degrade the neuron's responses to excitatory stimuli. Both within- and cross-modality effects were examined. An example of the former is when a response to a visual stimulus within its RF is suppressed by a second visual stimulus outside the RF. An example of the latter is when the response to a visual stimulus within the visual RF is suppressed when a stimulus from a different modality (e. g., auditory) is presented outside its (i.e., auditory) RF. Suppressive regions were found bordering visual, auditory, and somatosensory RFs. Despite significant modality-specific differences in the incidence and effectiveness of these regions, they were generally quite potent regardless of the modality. In the vast majority (85%) of cases, responses to the excitatory stimulus were degraded by >/=50% by simultaneously stimulating the suppressive region. Contrary to expectations and previous speculations, the effects of activating these suppressive regions often were quite specific. Thus powerful within-modality suppression could be demonstrated in many multisensory neurons in which cross-modality suppression could not be generated. However, the converse was not true. If an extra-RF stimulus inhibited center responses to stimuli of a different modality, it also would suppress center responses to stimuli of its own modality. Thus when cross-modality suppression was demonstrated, it was always accompanied by within-modality suppression. These observations suggest that separate mechanisms underlie within- and cross-modality suppression in the SC. Because some modality-specific tectopetal structures contain neurons with suppressive regions bordering their RFs, the within-modality suppression observed in the SC simply may reflect interactions taking place at the level of one input channel. However, the presence of modality-specific suppression at the level of one input channel would have no effect on the excitation initiated via another input channel. Given the modality-specificity of tectopetal inputs, it appears that cross-modality interactions require the convergence of two or more modality-specific inputs onto the same SC neuron and that the expression of these interactions depends on the internal circuitry of the SC. This allows a cross-modality suppressive signal to be nonspecific and to degrade any and all of the neuron's excitatory inputs.

The superior colliculus (SC) is thought to be unresponsive to stimuli that activate only short wavelength-sensitive cones (S-cones) in the retina. The apparent lack of S-cone input to the SC was recognized by Sumner et al. [Sumner, P., Adamjee, T., & Mollon, J. D. Signals invisible to the collicular and magnocellular pathways can capture visual attention. Current Biology, 12, 1312-1316, 2002] as an opportunity to test SC function. The idea is that visual behavior dependent on the SC should be impaired when S-cone stimuli are used because they are invisible to the SC. The SC plays a critical role in blindsight. If the SC is insensitive to S-cone stimuli blindsight behavior should be impaired when S-cone stimuli are used. Many clinical and behavioral studies have been based on the assumption that S-cone-specific stimuli do not activate neurons in the SC. Our goal was to test whether single neurons in macaque SC respond to stimuli that activate only S-cones. Stimuli were calibrated psychophysically in each animal and at each individual spatial location used in experimental testing [Hall, N. J., & Colby, C. L. Psychophysical definition of S-cone stimuli in the macaque. Journal of Vision, 13, 2013]. We recorded from 178 visually responsive neurons in two awake, behaving rhesus monkeys. Contrary to the prevailing view, we found that nearly all visual SC neurons can be activated by S-cone-specific visual stimuli. Most of these neurons were sensitive to the degree of S-cone contrast. Of 178 visual SC neurons, 155 (87%) had stronger responses to a high than to a low S-cone contrast. Many of these neurons' responses (56/178 or 31%) significantly distinguished between the high and low S-cone contrast stimuli. The latency and amplitude of responses depended on S-cone contrast. These findings indicate that stimuli that activate only S-cones cannot be used to diagnose collicular mediation.

The superior colliculus (SC) is a multilayered structure in the mammalian midbrain. Even though the behavioral function of the SC has been studied for well over 30 yr, it continues to be a source of surprising new findings. Traditionally it has been assumed that the primate SC selects and generates

It has long been believed that the superior colliculus (SC) is involved in the production of saccades but plays no role in the generation of vergence eye movements. However, results from several recent studies suggest that it may be worthwhile to examine the role of the SC in saccade-vergence interactions. Specifically, the available literature suggests two questions: do saccade-related neurons in SC have three-dimensional movement fields and is the slowing of saccades by vergence attributable, in part, to changes in the level of activity in SC? Single-unit data were recorded from 51 saccade-related neurons in rhesus monkey SC during saccades without vergence, saccades accompanied by convergence, and saccades accompanied by divergence. Most cells (78% for convergence, 86% for divergence) showed a significant reduction in peak spike density when the saccade was accompanied by vergence. A minority of cells (16% for convergence, 2% for divergence) increased their firing rate for saccades accompanied by vergence. Three cells were found that discharged in association with saccades, vergence, and the combination of the two. There were no cells that exhibited the pattern of discharge that would be expected of a cell tuned for saccades with divergence. Thus the present results do not support the hypothesis that saccade-related SC neurons are, as a rule, tuned in three dimensions. Small, but significant, differences in firing rate were often found for saccades without vergence at near and far distances. Approximately half of the cells showed a significant relationship between spike activity and saccade velocity, but the correlations tended to be very weak. This suggests that the decreased neuronal activity of SC neurons has only a limited effect on saccade velocity. For some cells, the movement field shifted for saccades with vergence. These shifts were highly variable from one cell to another.

Chapter 10 - Organization and plasticity in multisensory integration: early and late experience affects its governing principles

How the brain selects goals for movements remains unknown. The system designed to move the eyes rapidly, the saccadic system, may play a role. Here we ask how sensory signals within a saccade area are influenced by selecting and preparing a saccade. Trained monkeys made or withheld saccades, based on a color cue, to targets varying in luminance contrast. We measured the initial visual activity of superior colliculus (SC) neurons in response to the appearance of these targets. We determined neuronal contrast responses in three task conditions: when two luminance gratings appeared one in the response field (RF) and one in the mirror-opposite location and a cue to select the stimulus in the RF occurred; when the gratings appeared and a cue to select the stimulus out of the RF occurred; and third, when the gratings appeared but monkeys remained fixating on the central spot. SC neurons had increases in visual responses when contrast increased. Receiver operating characteristic analysis revealed an increased ability of neurons to detect the grating on trials with higher contrast targets and also on trials with a cue to make a saccade compared with trials with a cue to remain fixating. Using two measures of neuronal sensitivity, those SC neurons considered part of the motor circuitry increased their sensitivity to contrast with a cue to make a saccade. The results indicate that movement commands influence sensory responses in SC in much the same way that commands to shift attention influence sensory responses in extrastriate cortex.

Neurons in both the lateral intraparietal area (LIP) of the monkey parietal cortex and the intermediate layers of the superior colliculus (SC) are activated well in advance of the initiation of saccadic eye movements. To determine whether there is a progression in the covert processing for saccades from area LIP to SC, we systematically compared the discharge properties of LIP output neurons identified by antidromic activation with those of SC neurons collected from the same monkeys. First, we compared activity patterns during a delayed saccade task and found that LIP and SC neurons showed an extensive overlap in their responses to visual stimuli and in their sustained activity during the delay period. The saccade activity of LIP neurons was, however, remarkably weaker than that of SC neurons and never occurred without any preceding delay activity. Second, we assessed the dependence of LIP and SC activity on the presence of a visual stimulus by contrasting their activity in delayed saccade trials in which the presentation of the visual stimulus was either sustained (visual trials) or brief (memory trials). Both the delay and the presaccadic activity levels of the LIP neuronal sample significantly depended on the sustained presence of the visual stimulus, whereas those of the SC neuronal sample did not. Third, we examined how the LIP and SC delay activity relates to the future production of a saccade using a delayed GO/NOGO saccade task, in which a change in color of the fixation stimulus instructed the monkey either to make a saccade to a peripheral visual stimulus or to withhold its response and maintain fixation. The average delay activity of both LIP and SC neuronal samples significantly increased by the advance instruction to make a saccade, but LIP neurons were significantly less dependent on the response instruction than SC neurons, and only a minority of LIP neurons was significantly modulated. Thus despite some overlap in their discharge properties, the neurons in the SC intermediate layers showed a greater independence from sustained visual stimulation and a tighter relationship to the production of an impending saccade than the LIP neurons supplying inputs to the SC. Rather than representing the transmission of one processing stage in parietal cortex area LIP to a subsequent processing stage in SC, the differences in neuronal activity that we observed suggest instead a progressive evolution in the neuronal processing for saccades.

Movement fields of saccade-related burst neurons in the monkey superior colliculus

Complex visual scenes require that a target for an impending saccadic eye movement be selected from a larger number of possible targets. We investigated whether changing the probability that a visual stimulus would be selected as the target for a saccade altered activity of monkey superior colliculus (SC) neurons in two experiments. First, we changed the number of possible targets on each trial. Second, we kept the visual display constant and presented a single saccade target repeatedly so that target probability was established over time. Buildup neurons in the SC, those with delay period activity, showed a consistent reduction in activity as the probability of the saccade decreased, independent of the visual stimulus configuration. Other SC neurons, fixation and burst, were largely unaffected by the changes in saccade target probability. Because we had monkeys making saccades to many locations within the visual field, we could examine activity associated with saccades outside of the movement field of neurons. We found the activity of buildup neurons to be similar across the SC, before the target was identified, and reduced when the number of possible targets increased. The results of our experiments are consistent with a role for this activity in establishing a motor set. We found, consistent with this interpretation, that the activity of these neurons was predictive of the latency of a saccadic eye movement and not other saccade parameters such as end point or peak velocity.

Corticofugal projections to the thalamus reveal 2 axonal morphologies, each associated with specific physiological attributes. These determine the functional characteristics of thalamic neurons. It is not clear, however, whether such features characterize the corticofugal projections that mediate multisensory integration in superior colliculus (SC) neurons. The cortico-collicular projections from cat anterior ectosylvian sulcus (AES) are derived from its visual, auditory, and somatosensory representations and are critical for multisensory integration. Following tracer injections into each subdivision, 2 types of cortico-collicular axons were observed. Most were categorized as type I and consisted of small-caliber axons traversing long distances without branching, bearing mainly small boutons. The less frequent type II had thicker axons, more complex branching patterns, larger boutons, and more complex terminal boutons. Following combinatorial injections of 2 different fluorescent tracers into defined AES subdivisions, fibers from each were seen converging onto individual SC neurons and indicate that such convergence, like that in the corticothalamic system, is mediated by 2 distinct morphological types of axon terminals. Nevertheless, and despite the conservation of axonal morphologies across different subcortical systems, it is not yet clear if the concomitant physiological attributes described in the thalamus are directly applicable to multisensory integration.

Physiological and behavioral studies in cat have shown that corticotectal influences play important roles in the information-processing capabilities of superior colliculus (SC) neurons. While corticotectal inputs from the anterior ectosylvian sulcus (AES) play a comparatively small role in the unimodal responses of SC neurons, they are particularly important in rendering these neurons capable of integrating information from different sensory modalities (e.g., visual and auditory). The present experiments examined the behavioral consequences of depriving SC neurons of AES inputs, and thereby compromising their ability to integrate visual and auditory information. Selective deactivation of a variety of other cortical areas (posterolateral lateral suprasylvian cortex, PLLS; primary auditory cortex, AI; or primary visual cortex, 17/18) served as controls. Cats were trained in a perimetry device to ignore a brief, low-intensity auditory stimulus but to orient toward and approach a nearthreshold visual stimulus (a light-emitting diode, LED) to obtain food. The LED was presented at different eccentricities either alone (unimodal) or combined with the auditory stimulus (multisensory). Subsequent deactivation of the AES, with focal injections of a local anesthetic, had no effect on responses to unimodal cues regardless of their location. However, it profoundly, though reversibly, altered orientation and approach to multisensory stimuli in contralateral space. The characteristic enhancement of these responses observed when an auditory cue was presented in spatial correspondence with the visual stimulus was significantly degraded. Similarly, the inhibitory effect of a spatially disparate auditory cue was significantly ameliorated. The observed effects were specific to AES deactivation, as similar effects were not obtained with deactivation of PLLS, AI or 17/18, or saline injections into the AES. These observations are consistent with postulates that specific cortical-midbrain interactions are essential for the synthesis of multisensory information in the SC, and for the orientation and localization behaviors that depend on this synthesis.

1. Although a representation of multisensory space is contained in the superior colliculus, little is known about the spatial requirements of multisensory stimuli that influence the activity of neurons here. Critical to this problem is an assessment of the registry of the different receptive fields within individual multisensory neurons. The present study was initiated to determine how closely the receptive fields of individual multisensory neurons are aligned, the physiological role of that alignment, and the possible functional consequences of inducing receptive-field misalignment. 2. Individual multisensory neurons in the superior colliculus of anesthetized, paralyzed cats were studied with the use of standard extracellular recording techniques. The receptive fields of multisensory neurons were large, as reported previously, but exhibited a surprisingly high degree of spatial coincidence. The average proportion of receptive-field overlap was 86% for the population of visual-auditory neurons sampled. 3. Because of this high degree of intersensory receptive-field correspondence, combined-modality stimuli that were coincident in space tended to fall within the excitatory regions of the receptive fields involved. The result was a significantly enhanced neuronal response in 88% of the multisensory neurons studied. If stimuli were spatially disparate, so that one fell outside its receptive field, either a decreased response occurred (56%), or no intersensory effect was apparent (44%). 4. The normal alignment of the different receptive fields of a multisensory neuron could be disrupted by passively displacing the eyes, pinnae, or limbs/body. In no case was a shift in location or size observed in a neuron's other receptive field(s) to compensate for this displacement. The physiological result of receptive-field misalignment was predictable and based on the location of the stimuli relative to the new positions of their respective receptive fields. Now, for example, one component of a spatially coincident pair of stimuli might fall outside its receptive field and inhibit the other's effects. 5. These data underscore the dependence of multisensory integrative responses on the relationship of the different stimuli to their corresponding receptive fields rather than to the spatial relationship of the stimuli to one another. Apparently, the alignment of different receptive fields for individual multisensory neurons ensures that responses to combinations of stimuli derived from the same event are integrated to increase the salience of that event. Therefore the maintenance of receptive-field alignment is critical for the appropriate integration of converging sensory signals and, ultimately, elicitation of adaptive behaviors.

PROJECTION OF THE RETINA ON SUPERIOR COLLICULUS OF CATS