Artistic representations: Clues to efficient coding in human vision

BackgroundComparative studies of cognitive processes find similarities between humans and apes but also monkeys. Even high-level processes, like the ability to categorize classes of object from any natural scene under ultra-rapid time constraints, seem to be present in rhesus macaque monkeys (despite a smaller brain and the lack of language and a cultural background). An interesting and still open question concerns the degree to which the same images are treated with the same efficacy by humans and monkeys when a low level cue, the spatial frequency content, is controlled.Methodology/Principal FindingsWe used a set of natural images equalized in Fourier spectrum and asked whether it is still possible to categorize them as containing an animal and at what speed. One rhesus macaque monkey performed a forced-choice saccadic task with a good accuracy (67.5% and 76% for new and familiar images respectively) although performance was lower than with non-equalized images. Importantly, the minimum reaction time was still very fast (100 ms). We compared the performances of human subjects with the same setup and the same set of (new) images. Overall mean performance of humans was also lower than with original images (64% correct) but the minimum reaction time was still short (140 ms).ConclusionPerformances on individual images (% correct but not reaction times) for both humans and the monkey were significantly correlated suggesting that both species use similar features to perform the task. A similar advantage for full-face images was seen for both species. The results also suggest that local low spatial frequency information could be important, a finding that fits the theory that fast categorization relies on a rapid feedforward magnocellular signal.

Decision letter: Visual and motor signatures of locomotion dynamically shape a population code for feature detection in Drosophila

Color's contribution to rapid categorization of natural images is debated. We examine its effect on high-level face categorization responses using fast periodic visual stimulation (Rossion et al., 2015). A high-density electroencephalogram (EEG) was recorded during presentation of sequences of natural object images every 83 ms (i.e., at F = 12.0 Hz). Natural face images were embedded in the sequence at a fixed interval of F/9 (1.33 Hz). There were four conditions: (a) full-color images; (b) grayscale images; and (c) and (d) phase-scrambled images from Conditions 1 and 2, respectively, making faces and objects unrecognizable. Observers' task was to respond to color changes of the fixation cross (Experiment 1). We found face-categorization responses at 1.33 Hz and its harmonics (2.67 Hz, etc.) over occipitotemporal areas, with right-hemisphere dominance; responses to color images were not significantly different from those to grayscale images. Behavioral analysis revealed longer response times when images contained color, despite nearly-all-correct performance in all conditions, suggesting that color change in the task might detract from color's contribution to face categorization. We subsequently changed the task to responding to fixation shape changes so that such response-time differences were eliminated (Experiment 2). The aggregate face-categorization response became 21.6% stronger to color than to grayscale images. This color advantage occurred late, at 290-415 ms after stimulus onset. Our results suggest that the color advantage for face categorization interacts with behavior, and that color only has a moderate and relatively late contribution to rapid face categorization in natural images.

Signs are ubiquitous indoors and outdoors, which are used for way finding, finding shops and businesses, accessing variety of services. But the information of signs is inaccessible to many visually impaired people unless they are represented in a non-visual form such as Braille, tactile graphic, and speech. Automatic reading text from signs in natural images becomes a vital application in visually impaired people assistance. However, finding the text in scene images is a great challenge, because it cannot be assumed that the acquired image contains only characters. Natural scene images usually contain diverse complex text of different size, styles and colors with complex backgrounds. Therefore, this paper proposes a novel method for text extraction from scene images. The algorithm is implemented and evaluated using a set of natural scene images. Accuracy, precision and recall rates of the proposed method are analyzed to determine the success and limitation. Recommendations for improvements are given based on the results.

Considering the problem of automatic information acquisition in the field of intelligent transportation system (ITS), a new approach for detection of road traffic sign from natural scene images is proposed in this study. The adaptive colour segmentation based on pixel vector is firstly used to segment colour image into binary image and stand out traffic sign regions, which can reduce the influence of lighting conditions on image segmentation. Secondly, to improve the ability of shape identification during traffic sign detection, central projection transformation (CPT) is used to compute shape feature vectors of different candidate regions, and this shape feature is input to the probabilistic neural networks (PNN) to discriminate true traffic signs from candidates. The proposed approach is applied to many natural images. Experimental results show that the proposed method can effectively detect road traffic signs from natural scene images.

Information is one of the foremost fact in the prompt world. Within that, text information plays an imperative role and can acquire diverse mold. The natural images that consist of such text information are called scene text images. Semantic information of the image is used for content-based image retrieval, indexing and classification purpose. First stage of text extraction is the text and non-text classification that determines the presence of the text in an image. Compared to English language, determining the presence of Malayalam text in a scene image is more difficult due to its agglutinative nature. In this paper, the proposed work classifies natural scene images into Malayalam text and non-text images using Multiple Instance Learning (MIL). Our own dataset that contains natural scene images with Malayalam text and non-text images are used for the performance evaluation. Analysis is done in terms of precision, recall, F1 score, accuracy rates and has a promising experimental result.

The performance and effectiveness of CBIR scheme are directly associated with construction of small dimensional as well as salient image features respectively. So, in this paper, we have carried out the image retrieval process with small dimensional salient Image components or features as compare to the original image size and the imaze retrieval accuracy has been improved due to the consideration of local information rather than global information of Image data. Initially, the Image data is exploited in block level by discrete cosine transformation (DCT) and subsequently, some significant DCT coefficients are selected from each block as salient image components. However, all the DCT coefficients are not equally important in terms of visual perception and their dimension is also not negligible even in Image retrieval process. Later, selected AC coefficients are divided into four different groups. Later, some statistical parameters are computed from each group. Each statistical value is placed at a particular matrix. So, from each group, the number of matrices constructed is equal to the number of statistical parameters evaluated and one more matrix for DC coefficients is considered. Further, for construction of small dimensional feature vectors, gray level co-occurrence matrix (GLCM) is employed on all constructed matrices to derive the feature vector for a color component. The same procedure is employed on all three components and all feature vectors are combined toaether to form the final feature vector. The proposed CBIR structure is tested on three standard Image database i.e. Corel-lK, GHIM-10K and Olivia database and the experimental results demonstrate satisfactorily image retrieval and performance outperform other state-of-art schemes in many instances with respect to their precision values.

Eye movements in viewing urban images and natural images in diverse vegetation periods

The enhancement of Optical Character Recognition (OCR) is a lively research domain for numerous well-known applications. These applications are automation of logistics, reading sign boards, helping visually impaired subjects, and retrieval of textual content from natural scene images. The foremost objective of OCR in these applications is to acquire significant performance in order to convert scene text into machine-readable text with adequate accuracy. Henceforth, an improved technique is proposed based on image processing for segmenting textual content (foreground) from unstructured and unconstrained backgrounds. Further, a multimodal feature fusion and selection is carried out for classification. These features are serially fused and out passed to Crow Search Optimization (CSO) in order to generate a salient feature vector using entropy as fitness function. After that, the optimal features are used by ensemble classifiers for classification. The presented work is comprehensively tested on three mainstream challenging datasets SVT, MSRA-TD500, and KAIST consisting several indoor/outdoor natural images with harsh scenic variabilities. The experimental results with existing benchmark techniques confirm that the presented technique performs better in retrieving textual content. Similarly, in the classification process, the proposed technique works fine on different experiments due to the novel optimization method. Finally, the presented optimization method is compared with its counterparts with the same setup and it is found that it also performs better.

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features -- such as visual contrast, spatial scale, and depth -- differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.

We present a no-reference (NR) image quality assessment (IQA) algorithm that is inspired by the representation of visual scenes in the primary visual cortex of the human visual system. Specifically, we use the sparse coding model of the area V1 to construct an overcomplete dictionary for sparsely representing pristine (undistorted) natural images. First, we empirically demonstrate that the distribution of the sparse representation coefficients of natural images have sharp peaks and heavy tails, and can therefore be modeled using a Univariate Generalized Gaussian Distribution (UGGD). We then show that the UGGD model parameters form good features for distortion estimation and formulate our no-reference IQA algorithm based on this observation. Subsequently, we find UGGD model parameters that are representative of the class of pristine natural images. This is achieved using a training set of undistorted natural images. The perceptual quality of a test image is then defined to be the likelihood of its sparse coefficients being generated from the pristine UGGD model. We show that the proposed algorithm correlates well with subjective evaluation over several standard image databases. Further, the proposed method allows us to construct a distortion map that has several useful applications like distortion localization, adaptive rate allocation etc. Finally and importantly, the proposed NR-IQA algorithm does not make use of any distortion information or subjective scores during the training process.

Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject of study continuously. However, the way viewers perceive paintings is strictly related to the so-called HVS (Human Vision System) behaviour. This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. In further details, we introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans, including the fundamental understanding of a scene, and then extend it to painting images. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention. We use an FCNN (Fully Convolutional Neural Network), in which we exploit a differentiable channel-wise selection and Soft-Argmax modules. We also incorporate learnable Gaussian distributions onto the network bottleneck to simulate visual attention process bias in natural scene images. Furthermore, to reduce the effect of shifts between different domains (i.e. natural images, painting), we urge the model to learn unsupervised general features from other domains using a gradient reversal classifier. The results obtained by our model outperform existing state-of-the-art ones in terms of accuracy and efficiency.

The ratings of discomfort from a wide variety of images can be predicted from the energy at different spatial scales in the image, as measured by the Fourier amplitude spectrum of the luminance. Whereas comfortable images show the regression of Fourier amplitude against spatial frequency common in natural scenes, uncomfortable images show a regression with disproportionately greater amplitude at spatial frequencies within two octaves of 3 cycles deg(-1). In six studies, the amplitude in this spatial frequency range relative to that elsewhere in the spectrum explains variance in judgments of discomfort from art, from images constructed from filtered noise, and from art in which the phase or amplitude spectra have been altered. Striped patterns with spatial frequency within the above range are known to be uncomfortable and capable of provoking headaches and seizures in susceptible persons. The present findings show for the first time that, even in more complex images, the energy in this spatial-frequency range is associated with aversion. We propose a simple measurement that can predict aversion to those works of art that have reached the national media because of negative public reaction.

Primate visual systems process natural images in a hierarchical manner: at the early stage, neurons are tuned to local image features, while neurons in high-level areas are tuned to abstract object categories. Standard models of visual processing assume that the transition of tuning from image features to object categories emerges gradually along the visual hierarchy. Direct tests of such models remain difficult due to confounding alteration in low-level image properties when contrasting distinct object categories. When such contrast is performed in a classic functional localizer method, the desired activation in high-level visual areas is typically accompanied with activation in early visual areas. Here we used a novel image-modulation method called SWIFT (semantic wavelet-induced frequency-tagging), a variant of frequency-tagging techniques. Natural images modulated by SWIFT reveal object semantics periodically while keeping low-level properties constant. Using functional magnetic resonance imaging (fMRI), we indeed found that faces and scenes modulated with SWIFT periodically activated the prototypical category-selective areas while they elicited sustained and constant responses in early visual areas. SWIFT and the localizer were selective and specific to a similar extent in activating category-selective areas. Only SWIFT progressively activated the visual pathway from low- to high-level areas, consistent with predictions from standard hierarchical models. We confirmed these results with criterion-free methods, generalizing the validity of our approach and show that it is possible to dissociate neural activation in early and category-selective areas. Our results provide direct evidence for the hierarchical nature of the representation of visual objects along the visual stream and open up future applications of frequency-tagging methods in fMRI.

Computer Vision has become the poster child for Deep Learning. The image classification accuracy of convolutional neural nets on benchmark data sets has increased every year since their inception. This has been aided with advances in feature fusion. The increase in the availability of imagetext occurrence has lead to text augmented feature spaces that have lead to higher accuracy in in image classification tasks. However, these works are limited to instances where text is readily available. This study presents an approach to featurize text within natural images with the goal of augmenting image features for image classification tasks. Text extraction and featurization in natural images is a challenging task due to challenges in reliable text localization and OCR results, both being impeded by the variability in image text and errors in OCR. We overcome these challenges by implementing a novel bounding box concatenation algorithm and a novel feature boosting algorithm. The result is a pipeline that encodes an image into a text feature space. Classifiers trained on the text based feature space have comparable accuracy to the state of the art Convolutional Neural Nets (CNN's) while being significantly inexpensive computationally. Moreover, the augmentation of text features to image features generates a hybrid feature space with a higher information content for a classification problem when compared to a feature space comprised exclusively of image features. Thus, we see a rise in classification accuracy across all state of the art machine learning algorithms.