Suprathreshold visual psychophysics and structure-based visual masking

Adaptation to Natural Binocular Disparities in Primate V1 Explained by a Generalized Energy Model

PurposeAmblyopia is diagnosed as a reduced acuity in an otherwise healthy eye, which indicates that the deficit is not happening in the eye, but in the brain. One suspected mechanism explaining these deficits is an elevated amount of intrinsic blur in the amblyopic visual system compared to healthy observers. This “internally produced blur” can be estimated by the “equivalent intrinsic blur method”, which measures blur discrimination thresholds while systematically increasing the external blur in the physical stimulus. Surprisingly, amblyopes do not exhibit elevated intrinsic blur when measured with an edge stimulus. Given the fundamental ways in which they differ, synthetic stimuli, such as edges, are likely to generate contrasting blur perception compared to natural stimuli, such as pictures. Because our visual system is presumably tuned to process natural stimuli, testing artificial stimuli only could result in performances that are not ecologically valid.MethodsWe tested this hypothesis by measuring, for the first time, the perception of blur added to natural images in amblyopia and compared discrimination performance for natural images and synthetic edges in healthy and amblyopic groups.ResultsOur results demonstrate that patients with amblyopia exhibit higher levels of intrinsic blur than control subjects when tested on natural images. This difference was not observed when using edges.ConclusionsOur results suggest that intrinsic blur is elevated in the visual system representing vision from the amblyopic eye and that distinct statistics of images can generate different blur perception.

1. Luis H. Ospina, MD* 1. *Assistant Professor, Pediatric Ophthalmology and Neuro-ophthalmology, Ste-Justine Hospital, University de Montreal, Montreal, Quebec, Canada After completing this article, readers should be able to: 1. Recognize cortical visual impairment (CVI) as an important cause of pediatric visual loss. 2. Identify the most frequent causes of pediatric CVI and, more specifically, understand the role of hypoxia as a cause of central visual damage. 3. Describe the different patterns and implications for future vision of hypoxic central visual damage occurring in preterm compared with term babies. 4. Know the clinical signs suggestive of CVI in children. 5. Discuss the involvement of visual functions other than visual acuity (ie, cognitive visual aspects) in CVI and correlate them with the likely affected neuroanatomic substrates. 6. Explain the value of using a multidisciplinary approach in rehabilitating children who have CVI early in their development. Vision loss caused by central nervous system damage often is referred to as CVI but also may be referred to as cerebral visual impairment or neurologic visual impairment. These terms refer to the fact that the primary defect may not necessarily be limited to the striate (primary visual) cortex and may affect other areas subserving vision, such as the visual associative cortex, optic radiations, and visual attention pathways. Although it is not yet certain which of these three terms best describes the visual deficit, it is clear that use of the label cortical blindness must be abandoned. That term not only evokes negative connotations for the child and the family but is inaccurate because in almost every instance, some degree of residual vision remains, and visual improvement can occur. CVI has become the greatest cause of pediatric visual impairment in developed countries (1)(2)(3)(4)(5) and is becoming increasingly prevalent in developing nations. CVI is a problem of increasing frequency for two primary reasons. First, the progress in neonatal care has resulted in improved survival of …

Scientists are often asked to what extent a simple finding in a laboratory can be generalized to complicated phenomena in our daily lives. The same is equally true of vision science; numerous critical discoveries about our visual system have been made using very simple visual images, such as Gabor patches, but to what extent can these findings be applied to more natural images? Here, we used the fMRI decoding technique and directly tested whether the findings obtained with primitive visual stimuli (Gabor patches) were applicable to natural images. In the fMRI experiments, participants performed depth and resolution tasks with both Gabor patches and natural images. We created a fMRI decoder made from the results of the Gabor patch experiments that classified a brain activity pattern into the depth or resolution task, and then examined how successful the task-dependent decoder could sort a brain activity pattern in the natural image experiment into the depth or resolution task. As a result, we found that the task-dependent decoder constructed from Gabor patch experiments could predict which task (depth or resolution task) a participant was engaged in the natural image experiments, especially in the V3 and middle temporal (MT+) areas of the brain. This is consistent with previous researches on the cortical activation relating to depth perception rather than perceptual processing of display resolution. These results provide firm evidence that fMRI decoding technique possesses the power to evaluate the application of Gabor patch results (laboratory findings) to the natural images (everyday affairs), representing a new approach for studying the mechanism of visual perception.

The central goal of this dissertation is to design and model a smoothing filter based on the random single and mixed noise distribution that would attenuate the effect of noise while preserving edge details. Only then could robust, integrated and resilient edge detection methods be deployed to overcome the ubiquitous presence of random noise in images. Random noise effects are modeled as those that could emanate from impulse noise, Gaussian noise and speckle noise. In the first step, evaluation of methods is performed based on an exhaustive review on the different types of denoising methods which focus on impulse noise, Gaussian noise and their related denoising filters. These include spatial filters (linear, non-linear and a combination of them), transform domain filters, neural network-based filters, numerical-based filters, fuzzy based filters, morphological filters, statistical filters, and supervised learning-based filters. In the second step, switching adaptive median and fixed weighted mean filter (SAMFWMF) which is a combination of linear and non-linear filters, is introduced in order to detect and remove impulse noise. Then, a robust edge detection method is applied which relies on an integrated process including non-maximum suppression, maximum sequence, thresholding and morphological operations. The results are obtained on MRI and natural images. In the third step, a combination of transform domain-based filter which is a combination of dual tree – complex wavelet transform (DT-CWT) and total variation, is introduced in order to detect and remove Gaussian noise as well as mixed Gaussian and Speckle noise. Then, a robust edge detection is applied in order to track the true edges. The results are obtained on medical ultrasound and natural images. In the fourth step, a smoothing filter, which is a feed-forward convolutional network (CNN) is introduced to assume a deep architecture, and supported through a specific learning algorithm, l2 loss function minimization, a regularization method, and batch normalization all integrated in order to detect and remove impulse noise as well as mixed impulse and Gaussian noise. Then, a robust edge detection is applied in order to track the true edges. The results are obtained on natural images for both specific and non-specific noise-level.

The visual system needs to identify perceptually relevant borders to segment complex natural scenes. The primary visual cortex (V1) is thought to extract local borders, and higher visual areas are thought to identify the perceptually relevant borders between objects and the background. To test this conjecture, we used natural images that had been annotated by human observers who marked the perceptually relevant borders. We assessed the effect of perceptual relevance on V1 responses using human neuroimaging, macaque electrophysiology, and computational modeling. We report that perceptually relevant borders elicit stronger responses in the early visual cortex than irrelevant ones, even if simple features, such as contrast and the energy of oriented filters, are matched. Moreover, V1 neurons discriminate perceptually relevant borders surprisingly fast, during the early feedforward-driven activity at a latency of ~50 ms, indicating that they are tuned to the features that characterize them. We also revealed a delayed, contextual effect that enhances the V1 responses that are elicited by perceptually relevant borders at a longer latency. Our results reveal multiple mechanisms that allow V1 neurons to infer the layout of objects in natural images.

Experiments on visual detection in computer simulated noise (e.g. white noise) show that random variations from location to location in the image (due to noise) degrade human performance. Psychophysical experiments of visual detection of signals superimposed on a known deterministic background ('mask') show that human performance can be degraded by the presence of a high contrast deterministic background through divisive inhibition. The purpose of this paper is to perform a psychophysical experiment to determine the relative importance of these two sources of performance degradation (random background variations and contrast masking effects) in human visual detection in natural medical image backgrounds. The results show that both contrast masking and random background variations degrade human performance for detecting signals in natural medical image backgrounds. These results suggest that current observer models which do not include a source of degradation due to the deterministic presence of the background might need to model such effects in order to reliably predict human visual detection in natural medical image backgrounds.

This paper proposes an objective glossiness index for objects in halftone color images. In the proposed index, we consider the characteristics of the human visual system (HVS) and associate the image’s structure distortion and statistical information. According to the difference in the number of strategies adopted by the HVS in judging the difference between images, it is divided into single and multi-strategy modeling. In this study, we advocate multiple strategies to determine glossy or non-glossy quality. We assumed that HVS used different visual mechanisms to evaluate glossy and non-glossy objects. For non-glossy images, the image structure dominated, so the HVS tried to use structural information to judge distortion (a strategy based on structural distortion detection). For glossy images, the glossy appearance dominated; thus, the HVS tried to search for the glossiness difference (an appearance-based strategy). Herein, we present an index for glossiness assessment that attempts to explicitly model the structural dissimilarity and appearance distortion. We used the contrast sensitivity function to account for the mechanism of halftone images when viewed by the human eye. We estimated the structure distortion for the first strategy by using local luminance and contrast masking; meanwhile, local statistics changing in the spatial frequency components for skewness and standard deviation were used to estimate the appearance distortion for the second strategy. Experimental results showed that these two mixed-distortion measurement strategies performed well in consistency with the subjective ratings of glossiness in halftone color images.

Visual areas I and II of cerebral cortex of rabbit.

Image enhancement is a crucial pre-processing step for various image processing applications and vision systems. Many enhancement algorithms have been proposed based on different sets of criteria. However, a direct multi-scale image enhancement algorithm capable of independently and/or simultaneously providing adequate contrast enhancement, tonal rendition, dynamic range compression, and accurate edge preservation in a controlled manner has yet to be produced. In this paper, a multi-scale image enhancement algorithm based on a new parametric contrast measure is presented. The parametric contrast measure incorporates not only the luminance masking characteristic, but also the contrast masking characteristic of the human visual system. The formulation of the contrast measure can be adapted for any multi-resolution decomposition scheme in order to yield new human visual system-inspired multi-scale transforms. In this article, it is exemplified using the Laplacian pyramid, discrete wavelet transform, stationary wavelet transform, and dual-tree complex wavelet transform. Consequently, the proposed enhancement procedure is developed. The advantages of the proposed method include: 1) the integration of both the luminance and contrast masking phenomena; 2) the extension of non-linear mapping schemes to human visual system inspired multi-scale contrast coefficients; 3) the extension of human visual system-based image enhancement approaches to the stationary and dual-tree complex wavelet transforms, and a direct means of; 4) adjusting overall brightness; and 5) achieving dynamic range compression for image enhancement within a direct multi-scale enhancement framework. Experimental results demonstrate the ability of the proposed algorithm to achieve simultaneous local and global enhancements.

A new pyramidal JND (just noticeable difference) model is presented. The idea is to use this JND to determine the optimum strength for embedding the watermark providing an invisible and robust watermarking scheme. The image is first decomposed into a multiresolution representation using the pyramidal decomposition. Then, a perceptual model is proposed to compute the JND value for each pixel at each Laplacian level. This model takes into account three main characteristics of the human visual system (HVS), namely: contrast sensitivity, luminance adaptation and contrast masking. The performance of the proposed technique is evaluated in terms of transparency, using subjective and objective tests, and robustness to different common attacks.

Perceptual image quality metrics have explicitly accounted for perceptual characteristics of the human visual system (HVS) by modeling sensitivity to subband noise as the just-noticeable threshold of distortion. While such metrics can successfully account for contrast and luminance masking, they are quite sensitive to spatial shifts, intensity shifts, contrast changes, and scale changes. In contrast, the recently proposed structural similarity (SSIM) metrics account for perception more implicitly with the assumption that the HVS is adapted for extracting structural information (relative spatial covariance) from images. As such, they have the potential to be much more effective in quantifying suprathreshold compression artifacts than traditional perceptual metrics, as such artifacts tend to distort the structure of an image. We use a (perceptually) weighted variation of the complex wavelet SSIM (CWSSIM) to evaluate standard image compression techniques such as JPEG, JPEG 2000, SPIHT, and the Safranek-Johnston perceptual image coder. Our experimental results indicate that the weighted CWSSIM generally agrees with subjective evaluations.

Cultural heritage understanding and preservation is an important issue for society as it represents a fundamental aspect of its identity. Paintings represent a significant part of cultural heritage, and are the subject of study continuously. However, the way viewers perceive paintings is strictly related to the so-called HVS (Human Vision System) behaviour. This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. In further details, we introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans, including the fundamental understanding of a scene, and then extend it to painting images. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention. We use an FCNN (Fully Convolutional Neural Network), in which we exploit a differentiable channel-wise selection and Soft-Argmax modules. We also incorporate learnable Gaussian distributions onto the network bottleneck to simulate visual attention process bias in natural scene images. Furthermore, to reduce the effect of shifts between different domains (i.e. natural images, painting), we urge the model to learn unsupervised general features from other domains using a gradient reversal classifier. The results obtained by our model outperform existing state-of-the-art ones in terms of accuracy and efficiency.

A fundamental function of the visual system is to encode the building blocks of natural scenes-edges, textures and shapes-that subserve visual tasks such as object recognition and scene understanding. Essential to this process is the formation of abstract representations that generalize from specific instances of visual input. A common view holds that neurons in the early visual system signal conjunctions of image features, but how these produce invariant representations is poorly understood. Here we propose that to generalize over similar images, higher-level visual neurons encode statistical variations that characterize local image regions. We present a model in which neural activity encodes the probability distribution most consistent with a given image. Trained on natural images, the model generalizes by learning a compact set of dictionary elements for image distributions typically encountered in natural scenes. Model neurons show a diverse range of properties observed in cortical cells. These results provide a new functional explanation for nonlinear effects in complex cells and offer insight into coding strategies in primary visual cortex (V1) and higher visual areas.

The visual system is constantly bombarded with information originating from the outside world, but it is unable to process all the received information at any given time. In fact, the most salient parts of the visual scene are chosen to be processed involuntarily and immediately after the first glance along with endogenous signals in the brain. Vision scientists have shown that the early visual system, from retina to lateral geniculate nucleus (LGN) and then primary visual cortex, selectively processes the low-level features of the visual scene. Everything we perceive from the visual scene is based on these feature properties and their subsequent combination in higher visual areas. Different experiments have been designed to investigate the impact of these features on saliency and understand the relative visual mechanisms. In this paper, we review the psychophysical experiments which have been published in the last decades to indicate how the low-level salient features are processed in the early visual cortex and extract the most important and basic information of the visual scene. Important and open questions are discussed in this review as well and one might pursue these questions to investigate the impact of higher level features on saliency in complex scenes or natural images.