Dissociating stimulus information from internal representation—a case study in object recognition

THINGS-data reflects three large-scale neuroimaging and behavioral datasets of object processing in humans, comprising densely sampled functional MRI and magnetoencephalographic recordings, as well as 4.70 million similarity judgments in response to thousands of photographic images for up to 1854 objects.

Aphantasia and prosopagnosia are both rare conditions with impairments in visual cognition. While prosopagnosia refers to a face recognition deficit, aphantasics exhibit a lack of mental imagery. Current object recognition theories propose an interplay of perception and mental representations, making an association between recognition performance and visual imagery plausible. While the literature assumes a link between aphantasia and prosopagnosia, other impairments in aphantasia have been shown to be rather global. Therefore, we assumed that aphantasics do not solely exhibit impairments in face recognition but rather in general visual recognition performance, probably moderated by stimulus complexity. To test this hypothesis, 65 aphantasics were compared to 55 controls in a face recognition task, the Cambridge Face Memory Test, and a corresponding object recognition task, the Cambridge Car Memory Test. In both tasks, aphantasics performed worse than controls, indicating mild recognition deficits without face-specificity. Additional correlations between imagery vividness and performance in both tasks were found, suggesting that visual imagery influences visual recognition not only in imagery extremes. Stimulus complexity produced the expected moderation effect but only for the whole imagery-spectrum and only with face stimuli. Overall, the results imply that aphantasia is linked to a general but mild deficit in visual recognition.

Apperceptive Visual Agnosia: A Case Study

Deep neural networks have demonstrated exceptional performance in extracting task-specific representations from datasets, earning widespread recognition and application. However, the internal representations often reside in abstract, high-dimensional spaces that are unsupervised and difficult to interpret. Additionally, their complex and tightly coupled structures hinder researchers' ability to understand the models effectively. To tackle these challenges, we introduce NeuronExplorer, an analytical framework that employs self-supervised techniques for learning high-dimensional information representations. NeuronExplorer analyzes the high-dimensional representations derived from the basic units, namely neurons, within the neural network, predicting the clusters to which these neurons belong. This process facilitates the ‘community’ of neurons, enhancing interpretability.Moreover, we refine this neuron community structure by assessing the causal effects of intervening in neuron outputs, allowing us to measure the impact on model performance. NeuronExplorer ultimately enables a deeper understanding of the internal information representation within deep neural networks. Comprehensive experiments conducted across multiple models demonstrate that NeuronExplorer effectively mines internal representations, thereby improving model transparency.

Sequential movements rely on two information sources: external sensory cues and internal memory representations. Although often both sources jointly drive sequential behavior, previous research has primarily examined them in isolation. To address this, we trained participants (n = 26, 15F) to perform sequences of rapid finger presses in response to numerical cues. Sensory influence was measured by varying the number of visible cues, and memory influence was determined by comparing repeating and random sequences. Early in learning, participants integrated sensory and memory information: repeating sequences were performed more quickly when more cues were visible. After learning, when repeating sequences were predictable with certainty, participants relied solely on memory and ignored sensory cues. However, when this certainty was manipulated by introducing occasional violations within repeating sequences, participants reverted to integrating memory with sensory cues. We propose a computational model that successfully predicted both speed and accuracy of individual presses. Critically, this model relied on the assumption that multiple movements are planned independently of each other. This independence assumption was then validated by examining response patterns to isolated violations in repeating sequences. Finally, we provide evidence into how sequence memories can be flexibly deactivated and reactivated in response to these violations. Together, these results reveal how brain dynamically integrates sensory and memory information to produce sequences of movements.

Hippocampal firing is organized in theta sequences controlled by internal memory processes and by external sensory cues, but how these computations are coordinated is not fully understood. Although theta activity is commonly studied as a unique coherent oscillation, it is the result of complex interactions between different rhythm generators. Here, by separating hippocampal theta activity in three different current generators, we found epochs with variable theta frequency and phase coupling, suggesting flexible interactions between theta generators. We found that epochs of highly synchronized theta rhythmicity preferentially occurred during behavioral tasks requiring coordination between internal memory representations and incoming sensory information. In addition, we found that gamma oscillations were associated with specific theta generators and the strength of theta-gamma coupling predicted the synchronization between theta generators. We propose a mechanism for segregating or integrating hippocampal computations based on the flexible coordination of different theta frameworks to accommodate the cognitive needs.

We propose different methods for adaptively selecting information in images during object recognition. In contrast to standard feature selection, we consider this problem in a Bayesian framework where features are sequentially selected based on the current belief distribution over object classes. We define three different selection criteria and provide efficient Monte Carlo algorithms for the selection. In particular, we extend the successful Naive Bayes Nearest Neighbor (NBNN) classification approach, which is very costly to compute in its original form. We show that the proposed information selection methods result in a significant speed-up because only a small number of features needs to be extracted for accurate classification. In addition to adaptive methods based on the current belief distribution, we also consider image-based selection methods and we evaluate the performance of the different methods on a standard object recognition data set.

Information Processing and the Study of the Self

Object detection and recognition are fundamental capabilities for a mobile robot. Objects are a powerful representation for a variety of tasks including mobile manipulation and inventory tracking. As a result, object-based world representations have seen a great deal of research interest in the last several years. However, these systems usually assume that object recognition is well-solved: they require that accurate recognition be available for every object they might encounter. Despite steady advances, object recognition remains a difficult, open problem. Existing object recognition algorithms rely on high-resolution three-dimensional object models or on extensive hand-labeled training data. The sheer variety of objects that occur in natural environments makes manually training a recognizer for every possible object infeasible. In this work, we present a robotic system for unsupervised object and class discovery, in which objects are first discovered, and then grouped into classes in an unsupervised fashion. At each step, we approach the problem as one of robotics, not disembodied computer vision. On a very large robotic dataset, we discover object classes with 98.7% precision while achieving 71.8% recall. The scale and quality of these results demonstrate the merit of our approach, and prove the practicality of longterm large-scale object discovery. To our knowledge, no other authors have investigated robotic object discovery at this scale, making direct quantitative comparison impossible. We make our implementation and ground-truth labelings available, and evaluate our technique on a very large dataset. As a result, this work is a baseline against which future work can be compared.

Object recognition and generalisation during habituation in horses

Human visual object recognition is multifaceted and comprised of several domains of expertise. Developmental relations between young children's letter recognition and their 3-dimensional object recognition abilities are implicated on several grounds but have received little research attention. Here, we ask how preschoolers' success in recognizing letters relates to their ability to recognize 3-dimensional objects from sparse shape information alone. Seventy-three 2 1/2 to 5-year-old children completed a “Letter Recognition” task, measuring the ability to identify a named letter among 3 letters, and a “Shape Caricature Recognition” task, measuring recognition of familiar objects from sparse, abstract information about their part shapes and the spatial relations among those parts. Children also completed a control “Shape Bias” task, in which success depends on matching exact shapes but not on building an internal representation of the configuration of features characteristic of an object category or letter. Children's success in letter recognition was positively related to their shape caricature recognition scores, but not to their shape bias scores. The results suggest that letter recognition builds upon developing skills in attending to and representing the relational structure of object shape, and that these skills are common to both 2-dimensional and 3-dimensional object perception.

Behavioural, neural, and computational considerations suggest that the visual system may use (at least) two approaches to binding an object's features and/or parts into a coherent representation of shape: Dynamically bound (e.g., by synchrony of firing) representations of part attributes and spatial relations form a structural description of an object's shape, while units representing shape attributes at specific locations (i.e., a static binding of attributes to locations) form an analogue (image-like) representation of that shape. I will present a computational model of object recognition based on this proposal and empirical tests of the model. The model accounts for a large body of findings in human object recognition, and makes several novel and counter intuitive predictions. In brief, it predicts that visual priming for attended objects will be invariant with translation, scale, and left-right reflection, whereas priming for unattended objects will be invariant with translation and scale, but sensitive to left-right reflection. Five experiments demonstrated the predicted relationships between visual attention and patterns of visual priming as a function of variations in viewpoint. The implications of these findings for theories of visual binding and shape perception will be discussed.

Object classification has been one of the main tasks in computer vision. With the fast development of deep learning, its performance in image classification and object recognition has presented dramatic improvements. In this paper, we propose a new deep convolutional neural network (CNN) architecture for robust object classification. The proposed model is fused with three traditional CNN approaches, Densenet201, Resnet50, and our proposed residual CNN. The fused network architecture allows parallel processing of the multiple networks for keeping the system sped up. A single shot deep convolution network is trained as an object detector to generate all possible candidates of different object classes. The output of each neural network is representing a single vote that is used in the classification process. 3-to-1 voting criteria are applied in the final classification decision between the candidate object classes. Several experiments were conducted to evaluate the performance of the proposed network. The experimental results show that the proposed approach has better performance than the networks used in the fusion process when they act individually. It also has lower miss rates when compared to several states of art methodologies.

ABSTRACTIntroduction: Cross-modal interactions improve the processing of external stimuli, particularly when an isolated sensory modality is impaired. When information from different modalities is integrated, object recognition is facilitated probably as a result of bottom-up and top-down processes. The aim of this study was to investigate the potential effects of cross-modal stimulation in a case of simultanagnosia. Method: We report a detailed analysis of clinical symptoms and an 18F-fluorodeoxyglucose (FDG) brain positron emission tomography/computed tomography (PET/CT) study of a patient affected by Balint’s syndrome, a rare and invasive visual–spatial disorder following bilateral parieto-occipital lesions. An experiment was conducted to investigate the effects of visual and nonvisual cues on performance in tasks involving the recognition of overlapping pictures. Four modalities of sensory cues were used: visual, tactile, olfactory, and auditory. Results: Data from neuropsychological tests showed the presence of ocular apraxia, optic ataxia, and simultanagnosia. The results of the experiment indicate a positive effect of the cues on the recognition of overlapping pictures, not only in the identification of the congruent valid-cued stimulus (target) but also in the identification of the other, noncued stimuli. All the sensory modalities analyzed (except the auditory stimulus) were efficacious in terms of increasing visual recognition. Conclusions: Cross-modal integration improved the patient’s ability to recognize overlapping figures. However, while in the visual unimodal modality both bottom-up (priming, familiarity effect, disengagement of attention) and top-down processes (mental representation and short-term memory, the endogenous orientation of attention) are involved, in the cross-modal integration it is semantic representations that mainly activate visual recognition processes. These results are potentially useful for the design of rehabilitation training for attentional and visual–perceptual deficits.

Visually challenged are majorly dependent on the Braille language for comprehensive reading of textual documents and their walking sticks they hold everyday for their obstruction identification. Making them virtually visible in an environment and get them workable in a technical organisation this system innovates the technology which provides audio descriptions of their blind surroundings. The design involves human face, object and textual recognition which make vision for visually challenged. The smart kit contains an eye glass provided with camera, an ear phone, a microphone and the system where the processing is carried out. The camera present at the nose head of eye glass captures the intended image of the user as snapshots and transfer to the system where it gets processed and produces the specified audio descriptions as output. The system's database holds a corresponding text for each image which is then converted to audio stream when sounded. In case of any mismatch or no entry of the image in the datasets then the new image is stored as new dataset with the name specified through microphone. This avoids mismatch of that image in the future search.