Spatiotemporal Evidence Accumulation Through Saccadic Sampling for Object Recognition.

Decision letter: Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Our perception of an event is often modulated by our past experience and expectations [1Ramachandran V.S. Armel C. Foster C. Stoddard R. Object recognition can drive motion perception.Nature. 1998; 395: 852-853Crossref PubMed Scopus (35) Google Scholar, 2O'Regan J.K. Noe A. A sensorimotor account of vision and visual consciousness.Behav. Brain Sci. 2001; 24: 939-973Crossref PubMed Scopus (1863) Google Scholar]. Here we used a magic trick [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar] to demonstrate how magicians can distort our subjective perception and we investigate the mechanisms behind this deception. We found that when a magician performed an illusion in which a ball was seen to vanish in the air, 63% of observers perceived the ball leave his hand, move upwards, and disappear even though the ball did not leave the magician's hand. Moreover, observers' illusory perception of the ball was determined by cues that indicated the ball's location, namely the magician's head direction, rather than the percept itself. Furthermore, eye movement records revealed participants' strategic use of social cues prior to looking at the ball. Surprisingly, however, when the ball was not physically present, observers did not look at the area where they claimed to have seen the ball vanish, suggesting that the oculomotor system was not fooled by the illusion. These results show that although people's subjective percept is determined by expectations, the oculomotor system is largely driven by accurate bottom-up information, which is consistent with the suggestion that there are separate mechanisms for perception and visuomotor control [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar]. A striking example of magicians' deception is the vanishing ball illusion, in which a ball apparently disappears in mid air. This illusion is created by pretending to throw a ball up in the air, when in fact it remains secretly palmed in the magician's hand [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar]. If done convincingly, the observer should perceive the ball as moving up in the air even though it is no longer physically present. Similarly, when observers view a moving object that suddenly disappears, the final position of the object is usually perceived as being further along the path of motion than its actual final position [5Freyd J.J. Finke R.A. Representational Momentum.J. Exp. Psychol. Learn. Mem. Cogn. 1984; 10: 126-132Crossref Scopus (464) Google Scholar], a phenomenon known as representational momentum [6Assad J.A. Maunsell J.H.R. Neuronal correlates of inferred motion in primate posterior parietal carter.Nature. 1995; 373: 518-521Crossref PubMed Scopus (181) Google Scholar]. Much of the magician's deception relies on social cuing, in terms of his head and gaze direction, and we therefore predicted that the effectiveness of the vanishing ball illusion would be influenced by the magician's social cuing. Two different versions of the trick were created that addressed the extent to which the magician's social cues are responsible for the illusion. In the social cues pro-illusion condition, on the final 'fake' throw, the magician's eyes and head followed an imaginary ball moving upwards (Figure 1 and Supplemental data available on-line with this issue). In the social cues anti-illusion condition, the magic trick was identical except that on the final 'fake' throw, the magician looked at the hand concealing the ball, rather than following the imaginary ball (see Figure 2F). We also measured people's eye movements while they watched the video of the trick, to gain an online measure of the information obtained by the visual system. Immediately after having seen the magic trick, participants were questioned as to whether they perceived the 'illusory ball' moving towards the top of the screen on the final throw.Figure 2Participants' fixation points for the social cues pro-illusion (top row) and the social cues anti-illusion (bottom row) conditions.Show full captionThe dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002).View Large Image Figure ViewerDownload Hi-res image Download (PPT) The dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002). Participants who perceived the illusory ball had a vivid recollection of seeing the ball leave the screen at the top, and typically claimed that the illusion was created by someone catching it beyond the top of the screen. 68% of the participants in the social cues pro-illusion condition experienced the illusion, significantly more than in the social cues anti-illusion condition (32%, χ2 = 5.16, p = 0.025), thus demonstrating that the illusion was mediated by the social cuing. Most of the participants claimed that they spent their entire time looking at the ball. However, these reports deviated strongly from where they were actually looking. Figure 1 shows a timeline indicating where participants were looking during the magic trick. We found a very high consistency in the eye movement patterns between the participants, suggesting that people employed similar eye movement strategies. On throws where the ball was visible, participants generally looked at the ball once it reached the apex. But rather than merely tracking the ball, most of the participants glanced at the magician's face before looking at the ball, indicating that the visual system uses information about where the magician is looking as a way of predicting the location of the ball. Our results show that an observer's percept was driven by the magician's social cuing, and that participants utilized the magician's social cues. However, there was a surprising difference in the susceptibility of these two systems to the illusion. On the final illusory throw, most participants claimed to have seen the ball at the top of the screen. The oculomotor system on the other hand was not fooled by the illusion. Figure 2 shows that participants only looked at the top of the screen when the ball was physically present. These results illustrate a remarkable dissociation between what participants claimed to have seen and the way in which their eyes behaved. Whilst their percept was predominantly based on expectations, their eyes were largely driven by the actual visual input. Our results offer a striking example of how the dissociation between vision for perception and for action, well documented in neurological patients [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar, 7Goodale M.A. Haffenden A. Frames of reference for perception and action in the human visual system.Neurosci. Biobehav. Rev. 1998; 22: 161-172Crossref PubMed Scopus (110) Google Scholar, 8Milner A.D. Goodale M.A. The Visual Brain in Action. Oxford University Press, Oxford1995Google Scholar] and neurologically intact subjects [9Aglioti S. Desouza J.F.X. Goodale M.A. Size-contrast illusions deceive the eye but not the hand.Curr. Biol. 1995; 5: 679-685Abstract Full Text Full Text PDF PubMed Scopus (721) Google Scholar], also translates to more real world situations. We would like to thank the reviewers and Cristiana Cavina-Pratesi for their valuable comments on the manuscript, and Tim Smith for his help with a control experiment. Download .pdf (.1 MB) Help with pdf files Document S1. Supplemental Experimental Procedures Download .mpg (1.09 MB) Help with mpg files Movie 1 Vanishing ball trick pro-illusion condition Download .mpg (1.12 MB) Help with mpg files Movie 2 Vanishing ball trick anti-illusion condition

Perceiving, recognizing and remembering 3-dimensional (3-D) objects encountered in the environment has a very high survival value; unsurprisingly, this ability is shared among many animal species, including humans. The psychological, psychophysical and neural basis for object perception, discrimination, recognition and memory has been extensively studied in humans, monkeys, pigeons and rodents, but is still far from understood. Nearly all 3-D object recognition studies in the rodent used the “novel object recognition” paradigm, which relies on innate rather than learned behavior; however, this procedure has several important limitations. Recently, investigators have begun to recognize the power of behavioral tasks learned through reinforcement training (operant conditioning) to reveal the sensorimotor and cognitive abilities of mice and to elucidate their underlying neural mechanisms. Here, we describe a novel method for training and testing mice in visual and tactile object discrimination, recognition and memory, and use it to begin to examine the underlying sensory basis for these cognitive capacities. A custom-designed Y maze was used to train mice to associate one of two 3-D objects with a food reward. Out of nine mice trained in two cohorts, seven reached performance criterion in about 20–35 daily sessions of 20 trials each. The learned association was retained, or rapidly re-acquired, after a 6 weeks hiatus in training. When tested under low light conditions, individual animals differed in the degree to which they used tactile or visual cues to identify the objects. Switching to total darkness resulted only in a transient dip in performance, as did subsequent trimming of all large whiskers (macrovibrissae). Additional removal of the small whiskers (microvibrissae) did not degrade performance, but transiently increased the time spent inspecting the object. This novel method can be combined in future studies with the large arsenal of genetic tools available in the mouse, to elucidate the neural basis of object perception, recognition and memory.

In picture processing, picture segmentation, object classification and recognition is an impressive concept in present days because of increasing computer vision applications in real time applications. Based on required application, divide a picture into particular relevant clusters based on equivalent partitioning. Object identification is a minimum work to identify objects in a given picture or video with pixel values. The main representation of object classification from a given picture is to detect picture instances with respect to categorization of object. Object recognition is a complex task, when the pictures are poor quality, occlusion, and noise and in picture with background cutter sequences, this becomes more and efficient challenging task in picture processing. To appear object categorization and recognition effectively in pictures, there are several methods and approaches were introduced to solve object representation effectively. In this paper we review the different techniques of contextual feature extraction with respect to picture information of object classification, categorization and recognition of scalability and optimizations in real time picture processing applications. Our research gives better review of different approaches in picture categorization.

The vast growth of image databases creates many challenges for computer vision applications, for instance image retrieval and object recognition. Large variation in imaging conditions such as illumination and geometrical properties (including scale, rotation, and viewpoint) gives rise to the need for invariant features; i.e. image features should have minimal differences under these conditions. Local image features in the form of key points are widely used because of their invariant properties. In this chapter, we analyze different issues relating to existing local feature detectors. Based on this analysis, we present a new approach for detecting and filtering local features. The proposed approach is tested in a real-life application which supports navigation in urban environments based on visual information. The study shows that our approach performs as well as existing methods but with a significantly lower number of features.KeywordsFeature VectorLocal FeatureInterest PointQuery ImageImage RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Face recognition is considered as one of the relatively new and interesting concepts in the area of biometrics and comprises a huge number of applications. This study involves implementation of a robust recognition system by employing global and random local facial features of an individual. The proposed scheme considers the extraction of global facial features and some randomly selected local facial features. In order to exploit global information of a whole face image, Principal Component Analysis (PCA) algorithm is used. On the other hand, the randomly selected sub-images of a whole face image are concatenated and subsequently PCA is applied on the concatenated regions. In addition, the proposed scheme involves the optimization of random sub-image locations and sizes by considering different settings and choosing the most optimized one. Finally, Weighted Sum Rule fusion is employed to combine the calculated scores of the global and local feature extractors. The reliability of the proposed facial recognition system is investigated on several data sets of ORL, FERET, and Extended Yale face databases. Demonstration of results based on the recognition performance and ROC analysis clarifies that the proposed scheme achieves a considerable improvement compared to the global and local feature extractors implemented in this study.

THINGS-data reflects three large-scale neuroimaging and behavioral datasets of object processing in humans, comprising densely sampled functional MRI and magnetoencephalographic recordings, as well as 4.70 million similarity judgments in response to thousands of photographic images for up to 1854 objects.

Although visual object recognition is well studied and relatively well understood, much less is known about how shapes are recognized by touch and how such haptic stimuli might be compared with visual shapes. One might expect that the processes of visual and haptic object recognition engage similar brain structures given the advantages of avoiding redundant brain circuitry and indeed there is some evidence that this is the case. A potentially fruitful approach to understanding the differences in how shapes might be neurally represented is to find an algorithmic method of comparing shapes, which agrees with human behavior and determines whether that method differs between different modality conditions. If not, it would provide further evidence for a shared representation of shape. We recruited human participants to perform a one-back same–different visual and haptic shape comparison task both within (i.e., comparing two visual shapes or two haptic shapes) and across (i.e., comparing visual with haptic shapes) modalities. We then used various shape metrics to predict performance based on the shape, orientation, and modality of the two stimuli that were being compared on each trial. We found that the metrics that best predict shape comparison behavior heavily depended on the modality of the two shapes, suggesting differences in which features are used for comparing shapes depending on modality and that object recognition is not necessarily performed in a single, modality-agnostic region.

The visual recognition problem is central to computer vision research. From robotics to information retrieval, many desired applications demand the ability to identify and localize categories, places, and objects. This tutorial overviews computer vision algorithms for visual object recognition and image classification. We introduce primary representations and learning approaches, with an emphasis on recent advances in the field. The target audience consists of researchers or students working in AI, robotics, or vision who would like to understand what methods and representations are available for these problems. This lecture summarizes what is and isn't possible to do reliably today, and overviews key concepts that could be employed in systems requiring visual categorization. Table of Contents: Introduction / Overview: Recognition of Specific Objects / Local Features: Detection and Description / Matching Local Features / Geometric Verification of Matched Features / Example Systems: Specific-Object Recognition / Overview: Recognition of Generic Object Categories / Representations for Object Categories / Generic Object Detection: Finding and Scoring Candidates / Learning Generic Object Category Models / Example Systems: Generic Object Recognition / Other Considerations and Current Challenges / Conclusions

Developing a spatial searching tool to enhance the search capabilities of large spatial repositories for Geographical Information System (GIS) update has attracted more and more attention. Typically, objects to be detected are represented by many local features or local parts. Testing images are processed by extracting local features which are then matched with the object’s model image. Most existing work that uses local features assumes that each of the local features is independent to each other. However, in many cases, this is not true. In this paper, a method of applying the local cooccurring patterns to disclose the cooccurring relationships between local features for object detection is presented. Features including colour features and edge-based shape features of the interested object are collected. To reveal the cooccurring patterns among multiple local features, a colour cooccurrence histogram is constructed and used to search objects of interest from target images. The method is demonstrated in detecting swimming pools from aerial images. Our experimental results show the feasibility of using this method for effectively reducing the labour work in finding man-made objects of interest from aerial images.

Automatic age estimation from facial images is emerging as an important research area in recent years due to its promising potential for some computer vision applications. In this paper we propose a novel approach combine the global and local facial features in parallel manner to implement the age estimation. Then after extracting global and local features, these features are integrated for fine classification. In the proposed method, global and local features are extracted by Discrete Fourier Transform (DFT) and Principal Component Analysis (PCA) respectively. We have conducted experiments on a large scale age databases (FGNET). The experimental results are very promising in showing that it is an effective methodKeywordsage estimationglobal featurelocal featureclassifier integration

In the literature of psychophysics and neurophysiology, many studies have shown that both global and local features are crucial for face representation and recognition. This paper proposes a novel face recognition method which combines both global and local discriminative features. In this method, global features are extracted from whole face images by Fourier transform and local features are extracted from some spatially partitioned image patches by Gabor wavelet transform. After this, multiple classifiers are obtained by applying Fisher Discriminant Analysis on global Fourier features and local patches of Gabor features. All these classifiers are combined to form a hierarchical ensemble by sum rule. We evaluated the proposed method using Face Recognition Grand Challenge (FRGC) experimental protocols and database known as the largest data sets available. Experimental results on FRGC version 2.0 data set have shown that the proposed method achieves a verification rate of 86%, while the best reported was 76%.

One of the main issues of face recognition is to extract features from face images, which include both local and global features. We present a novel method to perform feature fusion at the feature level. First, global features are extracted by principal component analysis (PCA), while local features are obtained by active appearance model (AAM) and Gabor wavelet transform (GWT). Second, two types of features are fused by weighted concatenation. Finally, Euclidean and feature distances of fused features are applied to carry out a nearest neighbor classifier. The method is evaluated by the recognition rates and computation cost over two face image databases [AR (created by A. Martinez and R. Benavente) and SJTU-IPPR (Shanghai JiaoTong University-Institute of Image Processing and Pattern Recognition)]. Compared with PCA and elastic bunch graph matching (EBGM), the presented method is more effective. Though the recognition rate of the presented method is not as good as nonlinear feature combination (NFC), low computation cost is its superiority. In addition, experimental results show that the novel method is robust to variations over time, expression, illumination, and pose to a certain extent.

LocalFace: Learning significant local features for deep face recognition

Visual object recognition is subserved by ventral temporal and occipital regions of the brain. Regions comprising the dorsal visual pathway have not been considered relevant for object recognition, despite strong categorical biases for tool-related information in those regions. Here, we show that dorsal stream processes influence object categorization. We used two techniques to render prime pictures invisible: continuous flash suppression (CFS), which obliterates input into ventral temporal regions, but leaves dorsal stream processes largely unaffected, and backward masking (BM), which allows suppressed information to reach both ventral and dorsal stream structures. Categorically congruent primes suppressed under CFS facilitate categorization of tools but have no effect on nonmanipulable objects; in contrast, primes rendered invisible through BM facilitate target categorization for both tools and nonmanipulable things. Our findings demonstrate that information computed by the dorsal stream is used in object categorization, but only for a category of manipulable objects.