Object interaction space represented in scene-selective regions

A Real-World Size Organization of Object Responses in Occipitotemporal Cortex

The occipital place area represents the local elements of scenes

Human adults flawlessly and effortlessly navigate boundaries and obstacles in the immediately visible environment, a process we refer to as guided navigation. Neuroimaging work in adults suggests this ability involves the occipital place area (OPA) [1, 2]-a scene-selective region in the dorsal stream that selectively represents information necessary for visually guided navigation [3-9]. Despite progress in understanding the neural basis of visually guided navigation, however, little is known about how this system develops. Is navigationally relevant information processing present in the first few years of life? Or does this information processing only develop after many years of experience? Although a handful of studies have found selective responses to scenes (relative to objects) in OPA in childhood [10-13], no study has explored how more specific navigationally relevant information processing emerges in this region. Here, we do just that by measuring OPA responses to first-person perspective motion information-a proxy for the visual experience of actually navigating the immediate environment-using fMRI in 5- and 8-year-old children. We found that, although OPA already responded more to scenes than objects by age 5, responses to first-person perspective motion were not yet detectable at this same age and rather only emerged by age 8. This protracted development was specific to first-person perspective motion through scenes, not motion on faces or objects, and was not found in other scene-selective regions (the parahippocampal place area or retrosplenial complex) or a motion-selective region (MT). These findings therefore suggest that navigationally relevant information processing in OPA undergoes prolonged development across childhood.

The parahippocampal place area (PPA) is one of several brain regions that respond more strongly to scenes than to non-scene items such as objects and faces. The mechanism underlying this scene-preferential response remains unclear. One possibility is that the PPA is tuned to low-level stimulus features that are found more often in scenes than in less-preferred stimuli. Supporting this view, Nasr et al. (2014) recently observed that some of the stimuli that are known to strongly activate the PPA contain a large number of rectilinear edges. They further demonstrated that PPA response is modulated by rectilinearity for a range of non-scene images. Motivated by these results, we tested whether rectilinearity suffices to explain PPA selectivity for scenes. In the first experiment, we replicated the previous finding of modulation by rectilinearity in the PPA for arrays of 2-d shapes. However, two further experiments failed to find a rectilinearity effect for faces or scenes: high-rectilinearity faces and scenes did not activate the PPA any more strongly than low-rectilinearity faces and scenes. Moreover, the categorical advantage for scenes vs. faces was maintained in the PPA and two other scene-selective regions—the retrosplenial complex (RSC) and occipital place area (OPA)—when rectilinearity was matched between stimulus sets. We conclude that selectivity for scenes in the PPA cannot be explained by a preference for low-level rectilinear edges.

Perceived egocentric distance sensitivity and invariance across scene-selective cortex

Scene-selective regions (SSRs), including the parahippocampal place area (PPA), retrosplenial cortex (RSC), and transverse occipital sulcus (TOS), are among the most widely characterized functional regions in the human brain. However, previous studies have mostly focused on the commonality within each SSR, providing little information on different aspects of their variability. In a large group of healthy adults (N = 202), we used functional magnetic resonance imaging to investigate different aspects of topographical and functional variability within SSRs, including interindividual, interhemispheric, and sex differences. First, the PPA, RSC, and TOS were delineated manually for each individual. We then demonstrated that SSRs showed substantial interindividual variability in both spatial topography and functional selectivity. We further identified consistent interhemispheric differences in the spatial topography of all three SSRs, but distinct interhemispheric differences in scene selectivity. Moreover, we found that all three SSRs showed stronger scene selectivity in men than in women. In summary, our work thoroughly characterized the interindividual, interhemispheric, and sex variability of the SSRs and invites future work on the origin and functional significance of these variabilities. Additionally, we constructed the first probabilistic atlases for the SSRs, which provide the detailed anatomical reference for further investigations of the scene network. Hum Brain Mapp 38:2260-2275, 2017. © 2017 Wiley Periodicals, Inc.

Finding a path between locations is a routine task in daily life. Mental navigation is often used to plan a routeto a destination that is not visible from the current location. We first used functional magnetic resonance imaging (fMRI)and surface-based averaging methods to find high-level brain regions involved in imagined navigation between locationsin a building very familiar to each participant. This revealed a mental navigation network that includes the precuneus,retrosplenial cortex (RSC), parahippocampal place area (PPA), occipital place area (OPA), supplementary motor area(SMA), premotor cortex, and areas along the medial and anterior intraparietal sulcus. We then visualized retinotopic mapsin the entire cortex using wide-field, natural scene stimuli in a separate set of fMRI experiments. This revealed fivedistinct visual streams or ‘fingers’ that extend anteriorly into middle temporal, superior parietal, medial parietal,retrosplenial and ventral occipitotemporal cortex. By using spherical morphing to overlap these two data sets, we showedthat the mental navigation network primarily occupies areas that also contain retinotopic maps. Specifically, scene-selectiveregions RSC, PPA and OPA have a common emphasis on the far periphery of the upper visual field. Theseresults suggest that bottom-up retinotopic organization may help to efficiently encode scene and location information inan eye-centered reference frame for top-down, internally generated mental navigation. This study pushes the border ofvisual cortex further anterior than was initially expected.

The occipital place area (OPA) is a scene-selective region on the lateral surface of human occipitotemporal cortex that spatially overlaps multiple visual field maps, as well as portions of cortex that are not currently defined as retinotopic. Here we combined population receptive field modeling and responses to scenes in a representational similarity analysis (RSA) framework to test the prediction that the OPA's visual field map divisions contribute uniquely to the overall pattern of scene selectivity within the OPA. Consistent with this prediction, the patterns of response to a set of complex scenes were heterogeneous between maps. To explain this heterogeneity, we tested the explanatory power of seven candidate models using RSA. These models spanned different scene dimensions (Content, Expanse, Distance), low- and high-level visual features, and navigational affordances. None of the tested models could account for the variation in scene response observed between the OPA's visual field maps. However, the heterogeneity in scene response was correlated with the differences in retinotopic profiles across maps. These data highlight the need to carefully examine the relationship between regions defined as category-selective and the underlying retinotopy, and they suggest that, in the case of the OPA, it may not be appropriate to conceptualize it as a single scene-selective region.

The human entorhinal-hippocampal circuitry is characterized by an information-specific functional organization where two routes, that are preferentially connected to the parahippocampal cortex or the perirhinal and retrosplenial cortices, divide the entorhinal cortex as well as hippocampal subiculum and CA1 subregions.

We used a multi-voxel classification analysis of functional magnetic resonance imaging (fMRI) data to determine to what extent item-specific information about complex natural scenes is represented in several category-selective areas of human extrastriate visual cortex during visual perception and visual mental imagery. Participants in the scanner either viewed or were instructed to visualize previously memorized natural scene exemplars, and the neuroimaging data were subsequently subjected to a multi-voxel pattern analysis (MVPA) using a support vector machine (SVM) classifier. We found that item-specific information was represented in multiple scene-selective areas: the occipital place area (OPA), parahippocampal place area (PPA), retrosplenial cortex (RSC), and a scene-selective portion of the precuneus/intraparietal sulcus region (PCu/IPS). Furthermore, item-specific information from perceived scenes was re-instantiated during mental imagery of the same scenes. These results support findings from previous decoding analyses for other types of visual information and/or brain areas during imagery or working memory, and extend them to the case of visual scenes (and scene-selective cortex). Taken together, such findings support models suggesting that reflective mental processes are subserved by the re-instantiation of perceptual information in high-level visual cortex. We also examined activity in the fusiform face area (FFA) and found that it, too, contained significant item-specific scene information during perception, but not during mental imagery. This suggests that although decodable scene-relevant activity occurs in FFA during perception, FFA activity may not be a necessary (or even relevant) component of one's mental representation of visual scenes.

During real-world scene perception, viewers actively direct their attention through a scene in a controlled sequence of eye fixations. During each fixation, local scene properties are attended, analyzed, and interpreted. What is the relationship between fixated scene properties and neural activity in the visual cortex? Participants inspected photographs of real-world scenes in an MRI scanner while their eye movements were recorded. Fixation-related fMRI was used to measure activation as a function of lower- and higher-level scene properties at fixation, operationalized as edge density and meaning maps, respectively. We found that edge density at fixation was most associated with activation in early visual areas, whereas semantic content at fixation was most associated with activation along the ventral visual stream including core object and scene-selective areas (lateral occipital complex, parahippocampal place area, occipital place area, and retrosplenial cortex). The observed activation from semantic content was not accounted for by differences in edge density. The results are consistent with active vision models in which fixation gates detailed visual analysis for fixated scene regions, and this gating influences both lower and higher levels of scene analysis.

The perception and imagery of landmarks activate similar content‐dependent brain areas, including occipital and temporo‐medial brain regions. However, how these areas interact during visual perception and imagery of scenes, especially when recollecting their spatial location, remains unknown. Here, we combined functional magnetic resonance imaging (fMRI), resting‐state functional connectivity (rs‐fc), and effective connectivity to assess spontaneous fluctuations and task‐induced modulation of signals among regions entailing scene‐processing, the primary visual area and the hippocampus (HC), responsible for the retrieval of stored information. First, we functionally defined the scene‐selective regions, that is, the occipital place area (OPA), the retrosplenial complex (RSC) and the parahippocampal place area (PPA), by using the face/scene localizer, observing that two portions of the PPA—anterior and posterior PPA—were consistently activated in all subjects. Second, the rs‐fc analysis (n = 77) revealed a connectivity pathway similar to the one described in macaques, showing separate connectivity routes linking the anterior PPA with RSC and HC, and the posterior PPA with OPA. Third, we used dynamic causal modelling to evaluate whether the dynamic couplings among these regions differ between perception and imagery of familiar landmarks during a fMRI task (n = 16). We found a positive effect of HC on RSC during the retrieval of imagined places and an effect of occipital regions on both RSC and pPPA during the perception of scenes. Overall, we propose that under similar functional architecture at rest, different neural interactions take place between regions in the occipito‐temporal higher‐level visual cortex and the HC, subserving scene perception and imagery.

How do we understand the complex patterns of neural responses that underlie scene understanding? Studies of the network of brain regions held to be scene-selective—the parahippocampal/lingual region (PPA), the retrosplenial complex (RSC), and the occipital place area (TOS)—have typically focused on single visual dimensions (e.g., size), rather than the high-dimensional feature space in which scenes are likely to be neurally represented. Here we leverage well-specified artificial vision systems to explicate a more complex understanding of how scenes are encoded in this functional network. We correlated similarity matrices within three different scene-spaces arising from: (1) BOLD activity in scene-selective brain regions; (2) behavioral measured judgments of visually-perceived scene similarity; and (3) several different computer vision models. These correlations revealed: (1) models that relied on mid- and high-level scene attributes showed the highest correlations with the patterns of neural activity within the scene-selective network; (2) NEIL and SUN—the models that best accounted for the patterns obtained from PPA and TOS—were different from the GIST model that best accounted for the pattern obtained from RSC; (3) The best performing models outperformed behaviorally-measured judgments of scene similarity in accounting for neural data. One computer vision method—NEIL (“Never-Ending-Image-Learner”), which incorporates visual features learned as statistical regularities across web-scale numbers of scenes—showed significant correlations with neural activity in all three scene-selective regions and was one of the two models best able to account for variance in the PPA and TOS. We suggest that these results are a promising first step in explicating more fine-grained models of neural scene understanding, including developing a clearer picture of the division of labor among the components of the functional scene-selective brain network.

Detecting small objects is notoriously challenging due to their low resolution and noisy representation. Existing object detection pipelines usually detect small objects through learning representations of all the objects at multiple scales. However, the performance gain of such ad hoc architectures is usually limited to pay off the computational cost. In this work, we address the small object detection problem by developing a single architecture that internally lifts representations of small objects to "super-resolved" ones, achieving similar characteristics as large objects and thus more discriminative for detection. For this purpose, we propose a new Perceptual Generative Adversarial Network (Perceptual GAN) model that improves small object detection through narrowing representation difference of small objects from the large ones. Specifically, its generator learns to transfer perceived poor representations of the small objects to super-resolved ones that are similar enough to real large objects to fool a competing discriminator. Meanwhile its discriminator competes with the generator to identify the generated representation and imposes an additional perceptual requirement - generated representations of small objects must be beneficial for detection purpose - on the generator. Extensive evaluations on the challenging Tsinghua-Tencent 100K and the Caltech benchmark well demonstrate the superiority of Perceptual GAN in detecting small objects, including traffic signs and pedestrians, over well-established state-of-the-arts.

Recent research suggests that episodic memory is associated with systematic differences in the localization of neural activity observed during memory encoding and retrieval. The retrieval-related anterior shift is a phenomenon whereby the retrieval of a stimulus event (e.g., a scene image) is associated with a peak neural response which is localized more anteriorly than the response elicited when the stimulus is experienced directly. Here, we examine whether the magnitude of the anterior shift (i.e., the distance between encoding- and retrieval-related response peaks) is moderated by age, and also whether the shift is associated with memory performance. Younger and older human subjects of both sexes underwent fMRI as they completed encoding and retrieval tasks on word-face and word-scene pairs. We localized peak scene and face selectivity for each individual participant within the face-selective precuneus and in three scene-selective (parahippocampal place area [PPA], medial place area, occipital place area) ROIs. In line with recent findings, we identified an anterior shift in the PPA and occipital place area in both age groups and, in older adults only, in the medial place area and precuneus also. Of importance, the magnitude of the anterior shift was larger in older than in younger adults. The shift within the PPA exhibited an age-invariant across-participant negative correlation with source memory performance, such that a smaller displacement between encoding- and retrieval-related neural activity was associated with better performance. These findings provide novel insights into the functional significance of the anterior shift, especially in relation to memory decline in older age.SIGNIFICANCE STATEMENTCognitive aging is associated with reduced ability to retrieve precise details of previously experienced events. The retrieval-related anterior shift is a phenomenon in which category-selective cortical activity at retrieval is localized anterior to the peak activity at encoding. The shift is thought to reflect a bias at retrieval in favor of semantic and abstract information at the expense of low-level perceptual detail. Here, we report that the anterior shift is exaggerated in older relative to younger adults, and we demonstrate that a larger shift in the parahippocampal place area is associated with poorer memory performance. These findings suggest that the shift is sensitive to increasing age and that it is moderated by the quality and content of the retrieved episode.