English

Single-unit recordings demonstrated that the adult mammalian visual cortex is capable of reorganizing after induced retinal lesions. In humans, whether the adult cortex is capable of reorganizing has only been studied using functional magnetic resonance imaging, with equivocal results. Here, we exploited the phenomenon of visual crowding, a major limitation on object recognition, to show that, in humans with long-standing retinal (macular) lesions that afflict the fovea and thus use their peripheral vision exclusively, the signature properties of crowding are distinctly different from those of the normal periphery. Crowding refers to the inability to recognize objects when the object spacing is smaller than the critical spacing. Critical spacing depends only on the retinal location of the object, scales linearly with its distance from the fovea, and is approximately two times larger in the radial than the tangential direction with respect to the fovea, thus demonstrating the signature radial-tangential anisotropy of the crowding zone. Using retinal imaging combined with behavioral measurements, we mapped out the crowding zone at the precise peripheral retinal locations adopted by individuals with macular lesions as the new visual reference loci. At these loci, the critical spacings are substantially smaller along the radial direction than expected based on the normal periphery, resulting in a lower scaling of critical spacing with the eccentricity of the peripheral locus and a loss in the signature radial-tangential anisotropy of the crowding zone. These results imply a fundamental difference in the substrate of cortical processing in object recognition following long-term adaptation to macular lesions.

Image-based and structure-based matching processes in object recognition

Understanding the neural representation of spatial frequency (SF) in the primate cortex is vital for unraveling visual processing in object recognition. While many studies focus on SF in the primary visual cortex, the characteristics and interaction of SF with category representation remain unclear. To explore SF representation in the inferior temporal (IT) cortex of macaques, we conducted extracellular recordings with complex stimuli systematically filtered by SF. Our findings reveal explicit SF coding at both single-neuron and population levels. Temporal dynamics analysis of SF representation reveals that low SF (LSF) is decoded faster than high SF (HSF), and the SF preference dynamically shifts from LSF to HSF over time. Additionally, the SF representation for each neuron forms a profile that predicts category selectivity at the population level. IT neurons cluster into four groups based on SF preference, each with distinct category coding behaviors. Notably, HSF-preferring neurons show the highest category decoding for faces. Despite the existing connection between SF and category coding, we have identified uncorrelated representations of SF and category. Unlike category coding, SF is more sparsely represented and depends more on individual neurons. These findings dissociate SF and category representations, underscoring SF's pivotal role in object recognition.

Stereo matching is the key problem in many stereo vison based 3D applications. One of the factors make local stereo matching time-consuming is that every pixel has the same disparity range as the pre-set one, which should be larger than all possible disparities. An improper pre-set disparity range may lead to redundant computation for some pixels and inadequate computation for some others. In this paper, we propose an accurate and fast local stereo matching method, which employs fine disparity estimation and adaptive cost aggregation. The main contributions of our work include two parts. Firstly, we use phase-based correlation to estimate an initial disparity range for block center pixels. Secondly, we estimate a more limited disparity range for every in-block pixel according to its support weight to the block center pixel. Our disparity estimation and block matching techniques can not only reduce the disparity searching range for every pixel, but also can eliminate some pseudo match pairs. Four standard Middlebury stereo image pairs are tested to evaluate the performance of the proposed algorithm. Experimental results show that the proposed algorithm can reduce the matching time by 37.4% on average with relatively higher accuracy compared to traditional method.

In this paper, we address object recognition for a mobile robot which is deployed in a multistory building. To move to another floor, a mobile robot should recognize various objects related to an elevator, e.g., elevator control, call buttons, and LED displays. To this end, we propose a neural network based retrainable framework for object recognition, which consists of four components--preprocessing, binary classification, object identification, and outlier rejection. The binary classifier, a key component of our system, is a neural network that can be retrained, the motivation of which is to adapt to varying environments, especially with illuminations. Without incurring any extra process to prepare new training samples for retraining, they are freely obtained as a result of the outlier rejection component, being extracted on-line. To realize a practical system, we adopt a parallel architecture integrating both recognition and retraining processes for seamless object recognition, and furthermore detect and cope with the deterioration of a retrained neural network to ensure high reliability. We demonstrate the positive effect of retraining on the object recognition performance by conducting experiments over hundreds of images obtained in daytime and nighttime.

It is typically assumed that perception for action and object recognition are subserved by functionally and neuroanatomically distinct processing streams in the brain. However, recent evidence challenges this classical view and suggests an interaction between both visual processing streams. While previous studies showed an influence of object perception on action-related tasks, we investigated whether action representations facilitate visual object recognition. In order to address this question, two briefly displayed masked objects were sequentially presented, either affording congruent or incongruent motor interactions. We found superior naming accuracy for object pairs with congruent as compared to incongruent motor interactions (Experiment 1). This action priming effect indicates that action representations can facilitate object recognition. We further investigated the nature of the representations underlying this action priming effect. The effect was absent when the prime stimulus was presented as a word (Experiment 2). Thus, the action priming effect seems to rely on action representations specified by visual object information. Our findings suggest that processes of object-directed action influence object recognition.

Outlier detection and disparity refinement in stereo matching

A simultaneous multithreading multicore processor is proposed to accelerate object recognition for 720p HD video streams. The multithreading architecture with Q-learning based dynamic resource management enables concurrent processing of 8 region-of-interests with 5-stage fine grained recognition pipeline outperforming previous object recognition processors with 342GOPS computing power. In addition, the dynamic resource management contributes to increase of energy efficiency by applying the on-line learning DVFS and dynamic tile allocation based on task variance and hardware utilization to achieve 9.6mJ/frame with 1280×720 pixel image. It achieves 2.72× throughput and 3.7× energy efficiency compared to previous recognition processors.

Inspired by physiological and neuroimaging findings that revealed squint-induced modification of cortical volume and visual receptive field in early visual areas, we hypothesized that strabismic eyes without amblyopia manifest an increase in critical spacing of visual crowding, an essential bottleneck on object recognition and reliable psychophysical index of cortical organization. We used real-time eye tracking to ensure gaze-contingent display and examined visual crowding in patients with horizontal concomitant strabismus (both esotropia and exotropia) but without amblyopia and age-matched normal controls. Nineteen patients with exotropia (12 men, mean ± SD = 22.89 ± 7.82 years), 21 patients with esotropia (10 men, mean ± SD = 23.48 ± 6.95 years), and 14 age-matched normal controls (7 men, mean ± SD = 23.07 ± 1.07 years) participated in this study. We found that patients with strabismus without amblyopia showed significantly larger critical spacing with nasotemporal asymmetry in only the radial axis that related to the strabismus pattern, with exotropia exhibiting stronger temporal hemifield crowding and esotropia exhibiting stronger nasal hemifield crowding, in both the deviated and fixating eyes. Moreover, the magnitude of crowding change was related to the duration and degree of strabismic deviation. Using visual crowding as a psychophysical index of cortical organization, our study demonstrated significantly greater peripheral visual crowding with nasotemporal asymmetry in only the radial axis in patients with strabismus without amblyopia, indicating the existence of hemifield- and axis-specific miswiring of cortical processing in object recognition induced by long-term adaptation to ocular misalignment.

In this paper, a low-power real-time gaze-activated object recognition processor is proposed for a battery-powered smart glasses system. For high energy efficiency, we propose keypoint-level pipelined architecture to increase the hardware utilziation which results in significant power reduction of the real-time recognition processor. In addition, low-power gaze-activation image sensor with mixed-mode architecture is proposed for the glass user's gaze estimation. Therefore, only the small image region where the glasses user is seeing needs to be processed by the recognition processor leading to further power reduction. As a result, the proposed object recognition processor shows 30fps real-time performance only with 75mW power consumption, which is 3.5x and 4.4x smaller power than the state-of-the-art works.

A complete processing chain for visual object recognition is described in this paper. The system automatically detects individual objects on an assembly line, identifies their type, position, and orientation, and, thereby, forms the basis for automated object recognition and manipulation by single-arm robots. Two new ideas entered into the design of the recognition system. First we introduce a new fast and robust image segmentation algorithm that identifies objects in an unsupervised manner and describes them by a set of closed polygonal lines. Second we describe how to embed this object description into an object recognition process that classifies the objects by matching them to a given set of prototypes. Furthermore, the matching function allows us to determine the relative orientation and position of an object. Experimental results for a representative set of real-world tools demonstrate the quality and the practical applicability of our approach.KeywordsObject RecognitionShapeMesh GenerationModel SelectionRobotics

Background: Photographs provide more information including colour, luminance, texture, and shading cues compared to line drawings. There is evidence that these additional cues facilitate image recognition in individuals with and without neurological deficits. Black-and-white line drawings and colour photographs are commonly used with individuals with aphasia in evaluation and treatment. Eye tracking provides an opportunity to assess the influence of these different image types on object recognition processes without relying on verbal responses.Aims: The purpose of this study was to determine whether differences in object recognition of photographs and line drawings were observed using eye-tracking measures. Specific goals were to compare the influence of (a) image type of the same target image presented as colour photograph and black-and-white line drawing and (b) degree of colour diagnosticity (high-colour diagnostic natural objects and low-colour diagnostic human-made objects) in multiple-choice image displays on language-mediated eye movements in language-normal adults.Methods & Procedures: Eye movements of 19 participants were recorded while they viewed 66 multiple-choice image displays of colour photographs and black-and-white line drawings presented with a verbal stimulus. Target images included high-colour diagnostic natural objects and low-colour diagnostic human-made objects.Outcomes & Results: Participants allocated significantly greater proportions of fixation duration and first-pass gaze duration to the photographs compared to line drawings. No significant differences were observed for colour diagnosticity differences within the colour or the black-and-white line drawing displays.Conclusions: Eye tracking indexed significant differences in fixation duration allocated to target images in displays that only differed with respect to image type and colour, and were otherwise carefully controlled for shape, size, orientation, and content conveyed. Results suggest that language-normal participants’ object recognition was facilitated by colour photographs compared to line drawings, highlighting the clinical relevance and the need for research of image design for clinical use with individuals with neurological impairments.

Although visual object recognition is well studied and relatively well understood, much less is known about how shapes are recognized by touch and how such haptic stimuli might be compared with visual shapes. One might expect that the processes of visual and haptic object recognition engage similar brain structures given the advantages of avoiding redundant brain circuitry and indeed there is some evidence that this is the case. A potentially fruitful approach to understanding the differences in how shapes might be neurally represented is to find an algorithmic method of comparing shapes, which agrees with human behavior and determines whether that method differs between different modality conditions. If not, it would provide further evidence for a shared representation of shape. We recruited human participants to perform a one-back same–different visual and haptic shape comparison task both within (i.e., comparing two visual shapes or two haptic shapes) and across (i.e., comparing visual with haptic shapes) modalities. We then used various shape metrics to predict performance based on the shape, orientation, and modality of the two stimuli that were being compared on each trial. We found that the metrics that best predict shape comparison behavior heavily depended on the modality of the two shapes, suggesting differences in which features are used for comparing shapes depending on modality and that object recognition is not necessarily performed in a single, modality-agnostic region.

This paper proposes a stochastic approach to estimate the occlusion and disparity fields of stereoscopic images. The fields are estimated by the Bayesian maximum a posteriori (MAP) framework and Markov random field (MRF) models. The occlusion field model is based on the stochastic observation that the probability distribution in MAP estimator is relatively unstable and uniform at occluded regions. The occlusion is explicitly modelled as MRF, and is estimated in an energy optimization method called stochastic diffusion. The detected occluded region is compensated for the re-estimated disparity field in the same stochastic diffusion and the dynamic programming approach. Experimental results show good occlusion detection and disparity estimation. These results show the novel stochastic approach is suitable for occlusion detection and disparity estimation.

Warping-based spectral translation network for unsupervised cross-spectral stereo matching