The role of action representations in visual object recognition

Although visual object recognition is well studied and relatively well understood, much less is known about how shapes are recognized by touch and how such haptic stimuli might be compared with visual shapes. One might expect that the processes of visual and haptic object recognition engage similar brain structures given the advantages of avoiding redundant brain circuitry and indeed there is some evidence that this is the case. A potentially fruitful approach to understanding the differences in how shapes might be neurally represented is to find an algorithmic method of comparing shapes, which agrees with human behavior and determines whether that method differs between different modality conditions. If not, it would provide further evidence for a shared representation of shape. We recruited human participants to perform a one-back same–different visual and haptic shape comparison task both within (i.e., comparing two visual shapes or two haptic shapes) and across (i.e., comparing visual with haptic shapes) modalities. We then used various shape metrics to predict performance based on the shape, orientation, and modality of the two stimuli that were being compared on each trial. We found that the metrics that best predict shape comparison behavior heavily depended on the modality of the two shapes, suggesting differences in which features are used for comparing shapes depending on modality and that object recognition is not necessarily performed in a single, modality-agnostic region.

The primate visual system is often described as a hierarchical feature-conjunction pathway, whereby each level represents an increasingly complex combination of image elements, culminating in the representation of whole coherent images in anterior inferior temporal cortex. Although many models of the ventral visual stream emphasize serial feedforward processing ((Poggio T, Mutch J, Leibo J, Rosasco L, Tacchetti A. The computationalmagic of the ventral stream: sketch of a theory (and why some deep architectures work). TechRep MIT-CSAIL-TR-2012-035. MIT CSAIL, Cambridge, MA. 2012); (Yamins DLK, DiCarlo JJ. Eight open questions in the computational modeling of higher sensory cortex. Curr Opin Neurobiol. 2016:37:114-120.)), anatomical studies show connections that bypass intermediate areas and that feedback to preceding areas ((Distler C, Boussaoud D, Desimone R, Ungerleider LG. Cortical connections of inferior temporal area TEO in macaque monkeys. J Comp Neurol. 1993:334(1):125-150.); (Kravitz DJ, Saleem KS, Baker CI, Mishkin M. A new neural framework for visuospatial processing. Nat Rev Neurosci. 2011:12(4):217-230.)). Prior studies on visual discrimination and object transforms also provide evidence against a strictly feed-forward serial transfer of information between adjacent areas ((Kikuchi R, Iwai E. The locus of the posterior subdivision of the inferotemporal visual learning area in the monkey. Brain Res. 1980:198(2):347-360.); (Weiskrantz L, Saunders RC. Impairments of visual object transforms in monkeys. Brain. 1984:107(4):1033-1072.); (Kar K, DiCarlo JJ. Fast recurrent processing via ventrolateral prefrontal cortex is needed by the primate ventral stream for robust Core visual object recognition. Neuron. 2021:109(1):164-176.e5.)). Thus, we sought to investigate whether behaviorally relevant propagation of visual information is as strictly sequential as sometimes supposed. We compared the accuracy of visual recognition after selective removal of specific subregions of inferior temporal cortex-area TEO, area TE, or both areas combined. Removal of TEO alone had no detectable effect on recognition memory, whereas removal of TE alone produced a large and significant impairment. Combined removal of both areas created no additional deficit relative to removal of TE alone. Thus, area TE is critical for rapid visual object recognition, and detailed image-level visual information can reach area TE via a route other than through TEO.

Perceiving, recognizing and remembering 3-dimensional (3-D) objects encountered in the environment has a very high survival value; unsurprisingly, this ability is shared among many animal species, including humans. The psychological, psychophysical and neural basis for object perception, discrimination, recognition and memory has been extensively studied in humans, monkeys, pigeons and rodents, but is still far from understood. Nearly all 3-D object recognition studies in the rodent used the “novel object recognition” paradigm, which relies on innate rather than learned behavior; however, this procedure has several important limitations. Recently, investigators have begun to recognize the power of behavioral tasks learned through reinforcement training (operant conditioning) to reveal the sensorimotor and cognitive abilities of mice and to elucidate their underlying neural mechanisms. Here, we describe a novel method for training and testing mice in visual and tactile object discrimination, recognition and memory, and use it to begin to examine the underlying sensory basis for these cognitive capacities. A custom-designed Y maze was used to train mice to associate one of two 3-D objects with a food reward. Out of nine mice trained in two cohorts, seven reached performance criterion in about 20–35 daily sessions of 20 trials each. The learned association was retained, or rapidly re-acquired, after a 6 weeks hiatus in training. When tested under low light conditions, individual animals differed in the degree to which they used tactile or visual cues to identify the objects. Switching to total darkness resulted only in a transient dip in performance, as did subsequent trimming of all large whiskers (macrovibrissae). Additional removal of the small whiskers (microvibrissae) did not degrade performance, but transiently increased the time spent inspecting the object. This novel method can be combined in future studies with the large arsenal of genetic tools available in the mouse, to elucidate the neural basis of object perception, recognition and memory.

In recent years, the computer vision applications in the robotics have been improved to approach human-like visual perception and scene/context understanding. Following this aspiration, in this study, we explored the possibility of better object manipulation performance by connecting the visual recognition of objects to their physical attributes, such as weight and center of gravity (CoG). To develop and test this idea, an object manipulation platform is built comprising a robotic arm, a depth camera fixed at the top center of the workspace, embedded encoders in the robotic arm mechanism, and microcontrollers for position and force control. Since both the visual recognition and force estimation algorithms use deep learning principles, the test set-up was named as Deep-Table. The objects in the manipulation tests are selected from everyday life and are common to be seen on modern office desktops. The visual object localization and recognition processes are performed from two distinct branches by deep convolutional neural network architectures. We present five of the possible cases, having different levels of information availability on the object weight and CoG in the experiments. The results confirm that using our algorithm, the robotic arm can move different types of objects successfully varying from several grams (empty bottle) to around 250 g (ceramic cup) without failure or tipping. The proposed method also shows that connecting the object recognition with load estimation and contact point further improves the performance characterized by a smoother motion.

Visual prime stimuli can affect the processing of following target stimuli even if their visibility is reduced due to visual masking. Prime visibility depends on the stimulus parameters of the prime and those of the mask. Here we explored the effects of prime stimuli and modulated their visibility by continuous flash suppression (CFS). CFS reduces the visibility of a stimulus presented to one eye by simultaneously presenting a series of high-contrast masking stimuli to the other eye. We manipulated the strength of CFS effects on perception and examined how action priming effects of the masked stimuli varied under the same conditions. Prime visibility was modulated by the contrast of the primes (Experiments 1 and 2), the contrast of the masks (Experiments 2 and 3), and by the stimulus onset asynchrony between prime and target stimuli (all experiments). Surprisingly, action priming effects were modulated by these experimental variables in a parallel way. In addition, individual differences between participants in prime visibility correlated with individual differences in action priming. Our findings suggest that action priming and prime perception depend in similar ways on prime contrast, mask contrast, stimulus onset asynchrony, and individual dispositions in CFS. These findings distinguish CFS from other perceptual suppression techniques, such as backward masking, that allow reducing prime visibility without parallel effects on action priming. Our results corroborate the view that CFS interferes with visual processing at early stages in the cortical hierarchy with similar effects on later processing for perception and action.

BackgroundCognitive functional ability affects the accessibility of IT and is thus something that should be controlled for in user experience (UX) research. However, many cognitive function assessment batteries are long and complex, making them impractical for use in conventional experimental time frames. Therefore, there is a need for a short and reliable cognitive assessment that has discriminant validity for cognitive functions needed for general IT tasks. One potential candidate is the Trail Making Test (TMT).ObjectiveThis study investigated the usefulness of a digital TMT as a cognitive profiling tool in IT-related UX research by assessing its predictive validity on general IT task performance and exploring its discriminant validity according to discrete cognitive functions required to perform the IT task.MethodsA digital TMT (parts A and B) named Axon was administered to 27 healthy participants, followed by administration of 5 IT tasks in the form of CAPTCHAs (Completely Automated Public Turing tests to Tell Computers and Humans Apart). The discrete cognitive functions required to perform each CAPTCHA were rated by trained evaluators. To further explain and cross-validate our results, the original TMT and 2 psychological assessments of visuomotor and short-term memory function were administered.ResultsAxon A and B were administrable in less than 5 minutes, and overall performance was significantly predictive of general IT task performance (F5,19=6.352; P=.001; Λ=0.374). This result was driven by performance on Axon B (F5,19=3.382; P=.02; Λ=0.529), particularly for IT tasks involving the combination of executive processing with visual object and pattern recognition. Furthermore, Axon was cross-validated with the original TMT (Pcorr=.001 and Pcorr=.017 for A and B, respectively) and visuomotor and short-term memory tasks.ConclusionsThe results demonstrate that variance in IT task performance among an age-homogenous neurotypical population can be related to intersubject variance in cognitive function as assessed by Axon. Although Axon’s predictive validity seemed stronger for tasks involving the combination of executive function with visual object and pattern recognition, these cognitive functions are arguably relevant to the majority of IT interfaces. Considering its short administration time and remote implementability, the Axon digital TMT demonstrates the potential to be a useful cognitive profiling tool for IT-based UX research.

Object recognition is traditionally viewed as a hierarchical, bottom-up neural process. This view has been challenged recently by theoretical models and by findings indicating that top-down processes are involved in facilitating recognition. However, how such high-level information can be activated quickly enough to facilitate the bottom-up processing is yet unknown. We propose that such top-down facilitation is triggered by magnocellular information projected early and rapidly to the orbitofrontal cortex. Using human neuroimaging, we show that stimuli designed to bias processing toward the magnocellular pathway differentially activated the orbitofrontal cortex compared with parvocellular-biased stimuli. Although the magnocellular stimuli had a lower contrast than the parvocellular stimuli, they were recognized faster and just as accurately. Moreover, orbitofrontal activity predicted the performance advantage for the magnocellular, but not for the parvocellular-biased, stimuli, whereas the opposite was true in the fusiform gyrus. Last, analyses of effective connectivity using dynamic causal modeling showed that magnocellular-biased stimuli significantly activated pathways from occipital visual cortex to orbitofrontal cortex and from orbitofrontal cortex to fusiform gyrus. Conversely, parvocellular-biased stimuli significantly activated a pathway from the occipital visual cortex to fusiform gyrus. Our findings support the proposal that fast magnocellular projections linking early visual and inferotemporal object recognition regions with the orbitofrontal cortex facilitate object recognition by enabling the generation of early predictions.

A complete processing chain for visual object recognition is described in this paper. The system automatically detects individual objects on an assembly line, identifies their type, position, and orientation, and, thereby, forms the basis for automated object recognition and manipulation by single-arm robots. Two new ideas entered into the design of the recognition system. First we introduce a new fast and robust image segmentation algorithm that identifies objects in an unsupervised manner and describes them by a set of closed polygonal lines. Second we describe how to embed this object description into an object recognition process that classifies the objects by matching them to a given set of prototypes. Furthermore, the matching function allows us to determine the relative orientation and position of an object. Experimental results for a representative set of real-world tools demonstrate the quality and the practical applicability of our approach.KeywordsObject RecognitionShapeMesh GenerationModel SelectionRobotics

A single-case study is presented of a patient w ith a modality-specific problem in visual object recognition, w hich can be linked to impaired stored descriptions for objects. Performance was poor on tasks requiring naming, semantic decisions, and object decisions to seen objects. Performance on semantic decisions w ith w ords, however, was relatively good, demonstrating that the problem was not due solely to a general semantic impairment. Moreover, in contrast to the impairments in visual object recognition, both face recognition and on-line visual word recognition w ere within normal limits. The data indicate that im pairments of object recognition can be distinguished from impairments of face and visual w ord recognition, even when the problem can be localised within the structural description system for objects. We discuss the im plications of the results for understanding the relations between object, face, and word identification.

Deep neural networks inspired by the connections among biological neurons have been highly effective in the advancement of computer vision. Recent research into recurrent neural networks has taken account of forward as well as recurrent connections in a neural network architecture. In this study, we proposed a model that mirrors the architecture of the human ventral pathway with separate layers representing brain regions connected using long-range recurrent links. CIFAR-10/100 datasets were used to assess the performance of object recognition using the proposed model and the results were compared with those obtained using state-of-the-art methods. We demonstrated that the classification accuracy increased as the number of recurrences increased. Our results suggest that the proposed neuroscience-inspired model can facilitate object recognition in computer vision and may help to elucidate neurological mechanisms in the human brain.

Visual object recognition has been extensively studied under fixation conditions, but our natural viewing involves frequent saccadic eye movements that scan multiple local informative features within an object (e.g., eyes and mouth in a face image). These saccades would contribute to object recognition by subserving the integration of sensory information across local features, but mechanistic models underlying this process have yet to be established due to the presumed complexity of the interactions between the visual and oculomotor systems. Here, we employ a framework of perceptual decision making and show that human object categorization behavior with saccades can be quantitatively explained by a model that simply accumulates the sensory evidence available at each moment. Human participants of both sexes performed face and object categorization while they were allowed to freely make saccades to scan local features. Our model could successfully fit the data even during such a free viewing condition, departing from past studies that required controlled eye movements to test trans-saccadic integration. Moreover, further experimental results confirmed that active saccade commands (efference copy) do not substantially contribute to evidence accumulation. Therefore, we propose that object recognition with saccades can be approximated by a parsimonious decision-making model without assuming complex interactions between the visual and oculomotor systems.

Visual object recognition is just one of the many applications of camera-equipped smartphones. The ability to recognise objects through photos taken with wearable and handheld cameras is already possible through some of the larger internet search providers; yet, there is little rigorous analysis of the quality of search results, particularly where there is great disparity in image quality. This has motivated us to develop the Small Hand-held Object Recognition Test (SHORT). This includes a dataset that is suitable for recognising hand-held objects from either snapshots or videos acquired using hand-held or wearable cameras. SHORT provides a collection of images and ground truth that help evaluate the different factors that affect recognition performance. At its present state, the dataset is comprised of a set of high quality training images and a large set of nearly 135,000 smartphone-captured test images of 30 grocery products. In this paper, we will discuss some open challenges in the visual object recognition of objects that are being held by users. We evaluate the performance of a number of popular object recognition algorithms, with differing levels of complexity, when tested against SHORT.

Visual arts require the ability to process, categorize, recognize, and understand a variety of visual inputs. These challenges may engage and even influence mechanisms that are also relevant for visual object recognition beyond visual arts. A domain-general object recognition ability that applies broadly across a range of visual tasks was recently discovered. Here, we ask whether experience with visual arts is correlated with this domain-general ability. We developed a new survey to measure general visual arts experience and use it to measure arts experience in 142 individuals in whom we also estimated domain-general object recognition ability. Despite our measures demonstrating high reliability in a large sample size, we found substantial evidence (BF01 = 9.52) for no correlation between visual arts experience and general object recognition ability. This suggests that experience in visual arts has little influence on object recognition skills or vice versa, at least in our sample ranging from low to moderately high levels of arts experience. Our methods can be extended to other populations and our results should be replicated, as they suggest some limitations for the generalization of programs targeting visual literacy beyond the visual arts.

The role of invariant line junctions in object and visual word recognition

Natural objects, such as animals, flowers, and wood, as well as artificial objects, such as Japanese lacquerware, glass, and metal coins, have their own shape and surface appearances. Although object shapes play important roles in visual object recognition, the material properties or Shitsukan of objects are relevant for the recognition of visual objects. In this review, we introduce recent advances in the understanding of the neural basis of the cognition of Shitsukan. First, we introduce the results of single-neuron recording studies that examined neuronal activity in the inferior temporal cortex of macaque monkeys in response to the presentation of two-dimensional and three-dimensional shapes. Next, we introduce the studies of neuronal response property to the Shitsukan of objects, such as color, texture, glossiness, and the combinations of these features. Then, we discuss how the selective responses to Shitsukan are constructed by hierarchical processing in the ventral visual cortical pathway, which consists of V1, V2, and V4. Finally, we discuss the encoding of Shitsukan information in the inferior temporal cortex.