Perceived egocentric distance sensitivity and invariance across scene-selective cortex

The occipital place area represents the local elements of scenes

Human adults flawlessly and effortlessly navigate boundaries and obstacles in the immediately visible environment, a process we refer to as guided navigation. Neuroimaging work in adults suggests this ability involves the occipital place area (OPA) [1, 2]-a scene-selective region in the dorsal stream that selectively represents information necessary for visually guided navigation [3-9]. Despite progress in understanding the neural basis of visually guided navigation, however, little is known about how this system develops. Is navigationally relevant information processing present in the first few years of life? Or does this information processing only develop after many years of experience? Although a handful of studies have found selective responses to scenes (relative to objects) in OPA in childhood [10-13], no study has explored how more specific navigationally relevant information processing emerges in this region. Here, we do just that by measuring OPA responses to first-person perspective motion information-a proxy for the visual experience of actually navigating the immediate environment-using fMRI in 5- and 8-year-old children. We found that, although OPA already responded more to scenes than objects by age 5, responses to first-person perspective motion were not yet detectable at this same age and rather only emerged by age 8. This protracted development was specific to first-person perspective motion through scenes, not motion on faces or objects, and was not found in other scene-selective regions (the parahippocampal place area or retrosplenial complex) or a motion-selective region (MT). These findings therefore suggest that navigationally relevant information processing in OPA undergoes prolonged development across childhood.

The parahippocampal place area (PPA) is one of several brain regions that respond more strongly to scenes than to non-scene items such as objects and faces. The mechanism underlying this scene-preferential response remains unclear. One possibility is that the PPA is tuned to low-level stimulus features that are found more often in scenes than in less-preferred stimuli. Supporting this view, Nasr et al. (2014) recently observed that some of the stimuli that are known to strongly activate the PPA contain a large number of rectilinear edges. They further demonstrated that PPA response is modulated by rectilinearity for a range of non-scene images. Motivated by these results, we tested whether rectilinearity suffices to explain PPA selectivity for scenes. In the first experiment, we replicated the previous finding of modulation by rectilinearity in the PPA for arrays of 2-d shapes. However, two further experiments failed to find a rectilinearity effect for faces or scenes: high-rectilinearity faces and scenes did not activate the PPA any more strongly than low-rectilinearity faces and scenes. Moreover, the categorical advantage for scenes vs. faces was maintained in the PPA and two other scene-selective regions—the retrosplenial complex (RSC) and occipital place area (OPA)—when rectilinearity was matched between stimulus sets. We conclude that selectivity for scenes in the PPA cannot be explained by a preference for low-level rectilinear edges.

The perception and imagery of landmarks activate similar content‐dependent brain areas, including occipital and temporo‐medial brain regions. However, how these areas interact during visual perception and imagery of scenes, especially when recollecting their spatial location, remains unknown. Here, we combined functional magnetic resonance imaging (fMRI), resting‐state functional connectivity (rs‐fc), and effective connectivity to assess spontaneous fluctuations and task‐induced modulation of signals among regions entailing scene‐processing, the primary visual area and the hippocampus (HC), responsible for the retrieval of stored information. First, we functionally defined the scene‐selective regions, that is, the occipital place area (OPA), the retrosplenial complex (RSC) and the parahippocampal place area (PPA), by using the face/scene localizer, observing that two portions of the PPA—anterior and posterior PPA—were consistently activated in all subjects. Second, the rs‐fc analysis (n = 77) revealed a connectivity pathway similar to the one described in macaques, showing separate connectivity routes linking the anterior PPA with RSC and HC, and the posterior PPA with OPA. Third, we used dynamic causal modelling to evaluate whether the dynamic couplings among these regions differ between perception and imagery of familiar landmarks during a fMRI task (n = 16). We found a positive effect of HC on RSC during the retrieval of imagined places and an effect of occipital regions on both RSC and pPPA during the perception of scenes. Overall, we propose that under similar functional architecture at rest, different neural interactions take place between regions in the occipito‐temporal higher‐level visual cortex and the HC, subserving scene perception and imagery.

Neural responses to visual scenes reveals inconsistencies between fMRI adaptation and multivoxel pattern analysis

How do we understand the complex patterns of neural responses that underlie scene understanding? Studies of the network of brain regions held to be scene-selective—the parahippocampal/lingual region (PPA), the retrosplenial complex (RSC), and the occipital place area (TOS)—have typically focused on single visual dimensions (e.g., size), rather than the high-dimensional feature space in which scenes are likely to be neurally represented. Here we leverage well-specified artificial vision systems to explicate a more complex understanding of how scenes are encoded in this functional network. We correlated similarity matrices within three different scene-spaces arising from: (1) BOLD activity in scene-selective brain regions; (2) behavioral measured judgments of visually-perceived scene similarity; and (3) several different computer vision models. These correlations revealed: (1) models that relied on mid- and high-level scene attributes showed the highest correlations with the patterns of neural activity within the scene-selective network; (2) NEIL and SUN—the models that best accounted for the patterns obtained from PPA and TOS—were different from the GIST model that best accounted for the pattern obtained from RSC; (3) The best performing models outperformed behaviorally-measured judgments of scene similarity in accounting for neural data. One computer vision method—NEIL (“Never-Ending-Image-Learner”), which incorporates visual features learned as statistical regularities across web-scale numbers of scenes—showed significant correlations with neural activity in all three scene-selective regions and was one of the two models best able to account for variance in the PPA and TOS. We suggest that these results are a promising first step in explicating more fine-grained models of neural scene understanding, including developing a clearer picture of the division of labor among the components of the functional scene-selective brain network.

The Neurocognitive Basis of Spatial Reorientation

Previous work has shown that different cortical regions are sensitive to objects of differing real-world sizes (Konkle & Oliva, 2012; Mullally & Maguire, 2011; Bainbridge et al, 2012). Whereas a region in the occipito-temporal sulcus (OTS) shows small object selectivity, scene-selective regions (parahippocampal place area, PPA; retrosplenial cortex, RSC; and occipital place area, OPA) show systematically more activity for objects that are large in the world than smaller objects. What is the nature of this scene-like representation of objects? Here, we study an affordance-based object property: "interaction space". Interaction space is the area around objects that users traverse when interacting with the object. This interaction may include calculations of grasping points and surface area of contact, and is not necessarily conjoined with size. To fully deconvolve object interaction from physical size, we conducted a 2 x 2 factor design fMRI study with four conditions: 1) small one-handed objects (i.e., hairbrush), 2) large one-handed objects (i.e., umbrella), 3) small two-handed objects (i.e., hamburger), and 4) large two-handed objects (i.e., guitar). Stimuli were matched for ground truth size (based on physical sizes and weights), subjective size from an online survey, color, luminance, spectral energy, and retinal size. Results from a 2-factor ANOVA on beta values in functionally-defined ROIs show PPA, OPA, and RSC are sensitive to interaction space regardless of size, while PPA, OPA, and OTS are also sensitive to size regardless of interaction space. Importantly, effect sizes for interaction space in the PPA, OPA, and RSC were greater than those for object size, while the reverse was true for the OTS. These results show that interaction space is a perceptual object property that is represented in scene-selective regions, and suggest an overlapping interaction-based representation of objects and scenes in the brain. Meeting abstract presented at VSS 2013

Combined representation of visual features in the scene-selective cortex

The occipital place area (OPA) is a scene-selective region on the lateral surface of human occipitotemporal cortex that spatially overlaps multiple visual field maps, as well as portions of cortex that are not currently defined as retinotopic. Here we combined population receptive field modeling and responses to scenes in a representational similarity analysis (RSA) framework to test the prediction that the OPA's visual field map divisions contribute uniquely to the overall pattern of scene selectivity within the OPA. Consistent with this prediction, the patterns of response to a set of complex scenes were heterogeneous between maps. To explain this heterogeneity, we tested the explanatory power of seven candidate models using RSA. These models spanned different scene dimensions (Content, Expanse, Distance), low- and high-level visual features, and navigational affordances. None of the tested models could account for the variation in scene response observed between the OPA's visual field maps. However, the heterogeneity in scene response was correlated with the differences in retinotopic profiles across maps. These data highlight the need to carefully examine the relationship between regions defined as category-selective and the underlying retinotopy, and they suggest that, in the case of the OPA, it may not be appropriate to conceptualize it as a single scene-selective region.

We used a multi-voxel classification analysis of functional magnetic resonance imaging (fMRI) data to determine to what extent item-specific information about complex natural scenes is represented in several category-selective areas of human extrastriate visual cortex during visual perception and visual mental imagery. Participants in the scanner either viewed or were instructed to visualize previously memorized natural scene exemplars, and the neuroimaging data were subsequently subjected to a multi-voxel pattern analysis (MVPA) using a support vector machine (SVM) classifier. We found that item-specific information was represented in multiple scene-selective areas: the occipital place area (OPA), parahippocampal place area (PPA), retrosplenial cortex (RSC), and a scene-selective portion of the precuneus/intraparietal sulcus region (PCu/IPS). Furthermore, item-specific information from perceived scenes was re-instantiated during mental imagery of the same scenes. These results support findings from previous decoding analyses for other types of visual information and/or brain areas during imagery or working memory, and extend them to the case of visual scenes (and scene-selective cortex). Taken together, such findings support models suggesting that reflective mental processes are subserved by the re-instantiation of perceptual information in high-level visual cortex. We also examined activity in the fusiform face area (FFA) and found that it, too, contained significant item-specific scene information during perception, but not during mental imagery. This suggests that although decodable scene-relevant activity occurs in FFA during perception, FFA activity may not be a necessary (or even relevant) component of one's mental representation of visual scenes.

Recent research suggests that episodic memory is associated with systematic differences in the localization of neural activity observed during memory encoding and retrieval. The retrieval-related anterior shift is a phenomenon whereby the retrieval of a stimulus event (e.g., a scene image) is associated with a peak neural response which is localized more anteriorly than the response elicited when the stimulus is experienced directly. Here, we examine whether the magnitude of the anterior shift (i.e., the distance between encoding- and retrieval-related response peaks) is moderated by age, and also whether the shift is associated with memory performance. Younger and older human subjects of both sexes underwent fMRI as they completed encoding and retrieval tasks on word-face and word-scene pairs. We localized peak scene and face selectivity for each individual participant within the face-selective precuneus and in three scene-selective (parahippocampal place area [PPA], medial place area, occipital place area) ROIs. In line with recent findings, we identified an anterior shift in the PPA and occipital place area in both age groups and, in older adults only, in the medial place area and precuneus also. Of importance, the magnitude of the anterior shift was larger in older than in younger adults. The shift within the PPA exhibited an age-invariant across-participant negative correlation with source memory performance, such that a smaller displacement between encoding- and retrieval-related neural activity was associated with better performance. These findings provide novel insights into the functional significance of the anterior shift, especially in relation to memory decline in older age.SIGNIFICANCE STATEMENTCognitive aging is associated with reduced ability to retrieve precise details of previously experienced events. The retrieval-related anterior shift is a phenomenon in which category-selective cortical activity at retrieval is localized anterior to the peak activity at encoding. The shift is thought to reflect a bias at retrieval in favor of semantic and abstract information at the expense of low-level perceptual detail. Here, we report that the anterior shift is exaggerated in older relative to younger adults, and we demonstrate that a larger shift in the parahippocampal place area is associated with poorer memory performance. These findings suggest that the shift is sensitive to increasing age and that it is moderated by the quality and content of the retrieved episode.

Finding a path between locations is a routine task in daily life. Mental navigation is often used to plan a routeto a destination that is not visible from the current location. We first used functional magnetic resonance imaging (fMRI)and surface-based averaging methods to find high-level brain regions involved in imagined navigation between locationsin a building very familiar to each participant. This revealed a mental navigation network that includes the precuneus,retrosplenial cortex (RSC), parahippocampal place area (PPA), occipital place area (OPA), supplementary motor area(SMA), premotor cortex, and areas along the medial and anterior intraparietal sulcus. We then visualized retinotopic mapsin the entire cortex using wide-field, natural scene stimuli in a separate set of fMRI experiments. This revealed fivedistinct visual streams or ‘fingers’ that extend anteriorly into middle temporal, superior parietal, medial parietal,retrosplenial and ventral occipitotemporal cortex. By using spherical morphing to overlap these two data sets, we showedthat the mental navigation network primarily occupies areas that also contain retinotopic maps. Specifically, scene-selectiveregions RSC, PPA and OPA have a common emphasis on the far periphery of the upper visual field. Theseresults suggest that bottom-up retinotopic organization may help to efficiently encode scene and location information inan eye-centered reference frame for top-down, internally generated mental navigation. This study pushes the border ofvisual cortex further anterior than was initially expected.

Biologically inspired deep convolutional neural networks (CNNs), trained for computer vision tasks, have been found to predict cortical responses with remarkable accuracy. However, the internal operations of these models remain poorly understood, and the factors that account for their success are unknown. Here we develop a set of techniques for using CNNs to gain insights into the computational mechanisms underlying cortical responses. We focused on responses in the occipital place area (OPA), a scene-selective region of dorsal occipitoparietal cortex. In a previous study, we showed that fMRI activation patterns in the OPA contain information about the navigational affordances of scenes; that is, information about where one can and cannot move within the immediate environment. We hypothesized that this affordance information could be extracted using a set of purely feedforward computations. To test this idea, we examined a deep CNN with a feedforward architecture that had been previously trained for scene classification. We found that responses in the CNN to scene images were highly predictive of fMRI responses in the OPA. Moreover the CNN accounted for the portion of OPA variance relating to the navigational affordances of scenes. The CNN could thus serve as an image-computable candidate model of affordance-related responses in the OPA. We then ran a series of in silico experiments on this model to gain insights into its internal operations. These analyses showed that the computation of affordance-related features relied heavily on visual information at high-spatial frequencies and cardinal orientations, both of which have previously been identified as low-level stimulus preferences of scene-selective visual cortex. These computations also exhibited a strong preference for information in the lower visual field, which is consistent with known retinotopic biases in the OPA. Visualizations of feature selectivity within the CNN suggested that affordance-based responses encoded features that define the layout of the spatial environment, such as boundary-defining junctions and large extended surfaces. Together, these results map the sensory functions of the OPA onto a fully quantitative model that provides insights into its visual computations. More broadly, they advance integrative techniques for understanding visual cortex across multiple level of analysis: from the identification of cortical sensory functions to the modeling of their underlying algorithms.

Spatial knowledge about an environment can be cued from memory by perception of a visual scene during active navigation or by imagination of the relationships between nonvisible landmarks, such as when providing directions. It is not known whether these different ways of accessing spatial knowledge elicit the same representations in the brain. To address this issue, we scanned participants with fMRI, while they performed a judgment of relative direction (JRD) task that required them to retrieve real-world spatial relationships in response to either pictorial or verbal cues. Multivoxel pattern analyses revealed several brain regions that exhibited representations that were independent of the cues to access spatial memory. Specifically, entorhinal cortex in the medial temporal lobe and the retrosplenial complex (RSC) in the medial parietal lobe coded for the heading assumed on a particular trial, whereas the parahippocampal place area (PPA) contained information about the starting location of the JRD. These results demonstrate the existence of spatial representations in RSC, ERC, and PPA that are common to visually guided navigation and spatial imagery.