Deep supervised, but not unsupervised, models may explain IT cortical representation.

The inferior temporal (IT) cortex of the macaque monkey plays a pivotal role in the visual recognition of objects. In the IT cortex, a feature-selective network formed by connecting subregions specialized for common visual features seems to be a basic strategy for processing biologically important visual features. Gloss perception plays an important role in the judgment of materials and conditions of objects and is a biologically significant visual function. In the present study, we attempted to determine whether a neural circuit specialized for processing information related to gloss perception exists in the IT cortex in one monkey. We injected retrograde tracer into a gloss-selective subregion in the IT cortex where gloss-selective neurons were clustered in the neural recording experiment, and anatomically examined its neural connections. We observed that retrogradely labeled neurons were densely accumulated in multiple locations in the posterior and anterior IT cortices. Based on the results of this case study, we will discuss the possibility that, together with the injection site, the sites with a dense cluster of labeled neurons form feature-selective neural circuits for the processing of gloss information in the IT cortex.

Predictive analysis on cellular traffic is important for the control and monitoring of wireless networks. Cellular traffic prediction is a challenging problem due to the non- stationarity and dynamic spatial-temporal correlation of the traffic. In this paper, we address the problem of accurate traffic prediction in a base station by proposing a deep neural network called RAConv. Its structure includes residual network, attention mechanism, and deep convolutional network. In the proposed architecture, a deep 3D residual convolutional network (ResConv3D) with three residual blocks are employed to learn the local spatial-temporal features. An attention-aided convolutional long short-term memory network (AConvLSTM) is then used to capture the long-term spatial-temporal dependencies. The use of the attention modules enable the network to focus on the most important spatial-temporal information. We evaluate the performance of the proposed RAConv network using a dataset provided by a Canadian wireless service provider. We consider the traffic prediction on two time scales (i.e., hourly and daily), which exhibit different spatial-temporal dependency patterns. Experimental results show that the proposed RAConv network can achieve accurate prediction under both time scales. Results also show that our proposed network provides a lower root-mean-square error (RMSE) than the conventional ConvLSTM baseline scheme.

Understanding the neural representation of spatial frequency (SF) in the primate cortex is vital for unraveling visual processing in object recognition. While many studies focus on SF in the primary visual cortex, the characteristics and interaction of SF with category representation remain unclear. To explore SF representation in the inferior temporal (IT) cortex of macaques, we conducted extracellular recordings with complex stimuli systematically filtered by SF. Our findings reveal explicit SF coding at both single-neuron and population levels. Temporal dynamics analysis of SF representation reveals that low SF (LSF) is decoded faster than high SF (HSF), and the SF preference dynamically shifts from LSF to HSF over time. Additionally, the SF representation for each neuron forms a profile that predicts category selectivity at the population level. IT neurons cluster into four groups based on SF preference, each with distinct category coding behaviors. Notably, HSF-preferring neurons show the highest category decoding for faces. Despite the existing connection between SF and category coding, we have identified uncorrelated representations of SF and category. Unlike category coding, SF is more sparsely represented and depends more on individual neurons. These findings dissociate SF and category representations, underscoring SF's pivotal role in object recognition.

Large convolutional networks have achieved impressive classification performances recently. To achieve better performance, convolutional network tends to develop into deeper. However, the increase of network depth causes the linear growth of computational complexity, but cannot bring equivalent increase to the classification accuracy. To alleviate this inconsistence, we propose a cascading approach to accelerate the classification of very deep convolutional neural network. By exploiting the entropy metric to analyze the statistic differences of basic networks between the correctly and mistakenly classified images, we can assign the easily distinguished images to the shallow networks for reducing the computational complexity, and leave the difficultly classified images to the deep networks for maintaining the overall performance. Besides, the proposed cascaded networks can take advantage of the complementarity between different networks, which may boost the classification accuracy compared to the deepest network. We perform the experiments using residual networks of different depths on cifar100 dataset, on the condition of obtaining the similar accuracy to the deepest network, the results show that our cascaded ResNet32-ResNet110 and cascaded ResNet32-ResNet164 can reduce the computation time by 48.6% and 44.3% compared to ResNet110 and ResNet164, respectively. And the cascaded ResNet32-ResNet110-ResNet164 can reduce the computation time by 85.4% compared to the very deep Resnet1001.

Deep convolutional neural networks (DCNNs) trained for face identification can rival and even exceed human-level performance. The ways in which the internal face representations in DCNNs relate to human cognitive representations and brain activity are not well understood. Nearly all previous studies focused on static face image processing with rapid display times and ignored the processing of naturalistic, dynamic information. To address this gap, we developed the largest naturalistic dynamic face stimulus set in human neuroimaging research (700+ naturalistic video clips of unfamiliar faces). We used this naturalistic dataset to compare representational geometries estimated from DCNNs, behavioral responses, and brain responses. We found that DCNN representational geometries were consistent across architectures, cognitive representational geometries were consistent across raters in a behavioral arrangement task, and neural representational geometries in face areas were consistent across brains. Representational geometries in late, fully connected DCNN layers, which are optimized for individuation, were much more weakly correlated with cognitive and neural geometries than were geometries in late-intermediate layers. The late-intermediate face-DCNN layers successfully matched cognitive representational geometries, as measured with a behavioral arrangement task that primarily reflected categorical attributes, and correlated with neural representational geometries in known face-selective topographies. Our study suggests that current DCNNs successfully capture neural cognitive processes for categorical attributes of faces but less accurately capture individuation and dynamic features.

This paper proposes a method of accurately assessing image quality without a reference image by using a deep convolutional neural network. Existing training based methods usually utilize a compact set of linear filters for learning features of images captured by different sensors to assess their quality. These methods may not be able to learn the semantic features that are intimately related with the features used in human subject assessment. Observing this drawback, this work proposes training a deep convolutional neural network (CNN) with labelled images for image quality assessment. The ReLU in the CNN allows non-linear transformations for extracting high-level image features, providing a more reliable assessment of image quality than linear filters. To enable the neural network to take images of any arbitrary size as input, the spatial pyramid pooling (SPP) is introduced connecting the top convolutional layer and the fully-connected layer. In addition, the SPP makes the CNN robust to object deformations to a certain extent. The proposed method taking an image as input carries out an end-to-end learning process, and outputs the quality of the image. It is tested on public datasets. Experimental results show that it outperforms existing methods by a large margin and can accurately assess the image quality on images taken by different sensors of varying sizes.

The functional connectivity between cortical neurons is not static and is known to exhibit contextual modulations in terms of the coupling strength. Here we hypothesized that the information flow in a cortical local circuit exhibits complex forward-and-back dynamics, and conducted Granger causality analysis between the neuronal spike trains that were simultaneously recorded from macaque inferior temporal (IT) cortex while the animals performed a visual object discrimination task. Spikes from neuron pairs with a displaced peak on the cross-correlogram (CCG) showed Granger causality in the gamma-frequency range (30-80 Hz) with the dominance in the direction consistent with the CCG peak (forward direction). Although, in a classical view, the displaced CCG peak has been interpreted as an indicative of a pauci-synaptic serial linkage, temporal dynamics of the gamma Granger causality after stimulus onset exhibited a more complex triphasic pattern, with a transient forward component followed by a slowly developing backward component and subsequent reappearance of the forward component. These triphasic dynamics of causality were not explained by the firing rate dynamics and were not observed for cell pairs that exhibited a center peak on the CCG. Furthermore, temporal dynamics of Granger causality depended on the feature configuration within the presented object. Together, these results demonstrate that the classical view of functional connectivity could be expanded to incorporate more complex forward-and-back dynamics and also imply that multistage processing in the recognition of visual objects might be implemented by multiphasic dynamics of directional information flow between single neurons in a local circuit in the IT cortex.

We seek a general approach to determine what stimulus features visual neurons are sensitive to and how those features are represented by the neuron's responses. Because lesions of inferior temporal (IT) cortex interfere with a monkey's ability to perform pattern discrimination tasks we studied IT neurons. Previous single-unit studies have shown that IT neurons sometimes respond more strongly to complex stimuli (brushes, hands, faces) than to simple stimuli (bars, slits, edges). However, it is not known how specific stimulus parameters are represented by responses. We studied the responses of IT neurons in alert behaving monkeys to a large set of two-dimensional black and white patterns. The stimulus set was based on 64 Walsh functions that can be used to represent any picture with a resolution of one part in eight along each of two dimensions. The responses to these stimuli spanned a continuum from inhibition to strong excitation. A statistical test showed that the spike count was determined by which Walsh stimulus was presented. Hence, these stimuli form an adequate set for testing IT neurons. The responses showed temporal modulation of the spike train that could not be represented by a change in the spike count alone. Examples of this modulation were changes in latency, changes in the duration of the response, and alternating periods of excitation and inhibition. This temporal modulation may be important in representing stimulus parameters. The next paper in this series develops a method for quantifying this temporal modulation and shows that it is dependent on the stimulus. The third paper in this series shows that this temporal modulation contains more information about stimulus parameters than is contained in the spike count alone.

Local binary features for texture classification: Taxonomy and experimental study

Object detection systems based on the deep convolutional neural network (CNN) have recently made ground- breaking advances on several object detection benchmarks. While the features learned by these high-capacity neural networks are discriminative for categorization, inaccurate localization is still a major source of error for detection. Building upon high-capacity CNN architectures, we address the localization problem by 1) using a search algorithm based on Bayesian optimization that sequentially proposes candidate regions for an object bounding box, and 2) training the CNN with a structured loss that explicitly penalizes the localization inaccuracy. In experiments, we demonstrated that each of the proposed methods improves the detection performance over the baseline method on PASCAL VOC 2007 and 2012 datasets. Furthermore, two methods are complementary and significantly outperform the previous state-of-the-art when combined.

It has long been known that macaque inferior temporal (IT) neurons tend to fire more strongly to some shapes than to others, and that different IT neurons can show markedly different shape preferences. Beyond the discovery that these preferences can be elicited by features of moderate complexity, no general principle of (nonface) object recognition had emerged by which this enormous variation in selectivity could be understood. Psychophysical, as well as computational work, suggests that one such principle is the difference between viewpoint-invariant, nonaccidental (NAP) and view-dependent, metric shape properties (MPs). We measured the responses of single IT neurons to objects differing in either a NAP (namely, a change in a geon) or an MP of a single part, shown at two orientations in depth. The cells were more sensitive to changes in NAPs than in MPs, even though the image variation (as assessed by wavelet-like measures) produced by the former were smaller than the latter. The magnitude of the response modulation from the rotation itself was, on average, similar to that produced by the NAP differences, although the image changes from the rotation were much greater than that produced by NAP differences. Multidimensional scaling of the neural responses indicated a NAP/MP dimension, independent of an orientation dimension. The present results thus demonstrate that a significant portion of the neural code of IT cells represents differences in NAPs rather than MPs. This code may enable immediate recognition of novel objects at new views.

To reduce patient's dose, few-view CT reconstruction promises to be a good attempt. The key to better reconstruction is the sparse view artifacts. In recent years, DL(deep learing) has attracted a lot of attention because its outstanding performance in image processing. We propose a deep learning method for few-view CT reconstuction. Our method directly learns an end-to-end mapping between the full-view/few-view reconstruction. The mapping is represented as a deep convolutional neural network (CNN) that takes the few-view reconstruction image as the input and outputs the full-view one. We further show that traditional Dictionary Learning based reconstruction methods can also be viewed as a deep convolutional network. But unlike traditional methods that handle each component separately, our method jointly optimizes all layers. Our deep CNN has a lightweight structure, yet demonstrates state-of-the-art reconstruction quality, and achieves fast speed for practical on-line usage. We explore different network structures and parameter settings to achieve trade-offs between performance and speed.

According to the National Highway Traffic Safety Administration, one in ten fatal crashes and two in ten injury crashes were reported as distracted driver accidents in the United State during 2014. In an attempt to mitigate these alarming statistics, this paper explores using a dashboard camera along with computer vision and machine learning to automatically detect distracted drivers. We consider a dataset that incorporates drivers engaging in seven different distracting behaviors using left and/or right hands. Traditional handcrafted features paired with a Support Vector Machine classifier are contrasted with deep Convolutional Neural Networks. The traditional features include a blend of Histogram of Oriented Gradients and Scale-Invariant Feature Transform descriptors used to create Bags of Words. The deep convolutional methods use transfer learning on AlexNet, VGG-16, and ResNet-152. The results yield 85% accuracy with ResNet and 82.5% accuracy with VGG-16, which outperformed AlexNet by almost 10%. Replacing the fully connected layers by a Support Vector Machine classifier did not improve the classification accuracy. The traditional features yielded much lower accuracy than the deep convolutional networks.

Vehicle logo detection based on deep convolutional networks

This paper models unsupervised learning of an identity-based pattern (or copying) in speech called reduplication from raw continuous data with deep convolutional neural networks. We use the ciwGAN architecture (Beguš, 2021a) in which learning of meaningful representations in speech emerges from a requirement that the CNNs generate informative data. We propose a technique to wug-test CNNs trained on speech and, based on four generative tests, argue that the network learns to represent an identity-based pattern in its latent space. By manipulating only two categorical variables in the latent space, we can actively turn an unreduplicated form into a reduplicated form with no other substantial changes to the output in the majority of cases. We also argue that the network extends the identity-based pattern to unobserved data. Exploration of how meaningful representations of identity-based patterns emerge in CNNs and how the latent space variables outside of the training range correlate with identity-based patterns in the output has general implications for neural network interpretability.