Approaching the computational color constancy as a classification problem through deep learning

Colour is derived from three physical properties: incident light, object reflectance and sensor sensitivities. Incident light varies under natural conditions; hence, recovering scene illuminant is an important issue in computational colour. One way to deal with this problem under calibrated conditions is by following three steps, 1) building a narrow-band sensor basis to accomplish the diagonal model, 2) building a feasible set of illuminants, and 3) defining criteria to select the best illuminant. In this work we focus on colour constancy for natural images by introducing perceptual criteria in the first and third stages.To deal with the illuminant selection step, we hypothesize that basic colour categories can be used as anchor categories to recover the best illuminant. These colour names are related to how the human visual system has evolved to encode relevant natural colour statistics. Therefore the recovered image provides the best representation of the scene labelled with the basic colour terms. We demonstrate with several experiments how this selection criterion achieves current state-of-art results in computational colour constancy. In addition to this result, we psychophysically prove that usual angular error used in colour constancy does not correlate with human preferences, and we propose a new perceptual colour constancy evaluation.The implementation of this selection criterion strongly relies on the use of a diagonal model for illuminant change. Then, the second contribution focuses on building an appropriate narrow-band sensor basis to represent natural images. We propose to use the spectral sharpening technique to compute a unique narrow-band basis optimized to represent a large set of natural reflectances under natural illuminants and given in the basis of human cones. The proposed sensors allow predicting unique hues and the World colour Survey data independently of the illuminant by using a compact singularity function. Additionally, we studied different families of sharp sensors to minimize different perceptual measures. This study brought us to extend the spherical sampling procedure from 3D to 6D.Several research lines remain still open, such as, measuring the effects of using the computed sharp sensors on the category hypothesis; or inserting spatial contextual information to improve category hypothesis. Finally, to explore how individual sensors can be adjusted to the colours in a scene.

Color constancy is of important for many computer vision applications, such as image classification, color object recognition, object tracking and so on. But unlike the human visual system, imaging device cannot be able to compute color constant descriptors which do not vary with the color of the illuminant, so solving color constancy problem is necessary. In the calculation of color constancy, illuminant estimation is the key. Because grey surfaces can perfectly reflect the color of the scene illumination, many methods have been proposed to identify grey surfaces to estimate the illuminant. But they either rely on the camera.s parameters, lacking universality, or work inaccurate in worse conditions. In order to solve these problems, in this paper, an iterative method is proposed. The quality of the proposed method is tested and compared to the previous color constancy methods on the Macbeth Chart and two data sets of synthetic and real images. Through MATLAB simulation, experimental pictures and quantitative data for performance evaluation were gotten. The simulated results show that the proposed algorithm is accurate and efficient in identification of the grey surfaces, even in worse condition. And it performs well in color constancy computation on both synthetic and real images.

Dermoscopy images classification is the major method for melanoma detection. Currently, the most effective method of image classification is deep learning. However, training a deep neural network requires huge amounts of training data, and it is difficult to collect enough dermoscopy images. In this paper, we introduce the deep attention network which is optimized by attention mechanism and fisher criterion. Experiment results showed that this hybrid neural network improved the recognition rate when sample size was insufficient. Besides, considering the color of dermoscopy image is an important factor for melanoma detection, and the illumination may alter the color of dermoscopy image. We researched the color constancy algorithms for color calibration and tested the impact of color constancy on our model. The result proved that color constancy can further improve the recognition rate.

In many multi-media applications it is desirable to separate the influence of the illumination sources and imaging equipment from the properties of the depicted scene. The ability of the human visual system to solve this task in many situations is known as color constancy. Technical applications of these methods include automatic color correction and illumination independent search in image databases. Many conventional computational color constancy methods assume that the effect of an illumination change can be described by a matrix multiplication with a diagonal matrix. In this paper we introduce a color normalization algorithm which computes the unique color transformation matrix which normalizes a given set of moments computed from the color distribution of an image. This normalization procedure is a generalization of the channel independent color constancy methods since general matrix transformations are considered. We compare the performance of this new normalization method with conventional color constancy methods. The experiments show that diagonal transformation matrices provide a better illumination compensation. This shows that the color moments also contain significant information about the color distributions of the objects in the image which is independent of the illumination characteristics. In another set of experiments we use the unique transformation matrix as a descriptor of the set of moments which describe the global color distribution in the image. Combining the matrices computed from two such images describes the color differences between them. We then use this as a tool for color dependent search in image databases. This matrix based color search is computationally less demanding than histogram based color search tools.

Color constancy is the ability to assign a constant color to an object independent of changes in illumination. Color constancy is achieved by taking context information into account. Previous approaches that have used matching paradigms to quantify color constancy found degrees of constancy between 20% and 80%. Here, we studied color constancy in a color-naming task under different conditions of surround illumination and patch size. Observers categorized more than 400 patches for each illumination condition. This allows one to overcome inherent limitations in color naming and to study the changes in color categories under illumination changes. When small central test patches with a full context illumination were categorized, observers followed the illumination shift almost completely, showing a high degree of constancy (99%). Reducing the available context information or increasing the patch size decreased the degree of constancy to about 50%. Moderate degrees of constancy (66%) occurred even when the test patches were never viewed simultaneously but only in temporal alternation with the illumination. Boundaries between color categories were largely stable within and across observers under neutral illumination. Under changing illumination, there were small but systematic variations in the color category boundaries. Color category boundaries tended to rotate away from the illumination color. This variation was largest under full context conditions where highest degrees of color constancy were obtained.

Visual information, in the form of images and video, comes from the interaction of light with objects. Illumination is a fundamental element of visual information. Detecting and interpreting illumination effects is part of our everyday life visual experience. Shading for instance allows us to perceive the three-dimensional nature of objects. Shadows are particularly salient cues for inferring depth information. However, we do not make any conscious or unconscious effort to avoid them as if they were an obstacle when we walk around. Moreover, when humans are asked to describe a picture, they generally omit the presence of illumination effects, such as shadows, shading, and highlights, to give a list of objects and their relative position in the scene. Processing visual information in a way that is close to what the human visual system does, thus being aware of illumination effects, represents a challenging task for computer vision systems. Illumination phenomena interfere in fact with fundamental tasks in image analysis and interpretation applications, such as object extraction and description. On the other hand, illumination conditions are an important element to be considered when creating new and richer visual content that combines objects from different sources, both natural and synthetic. When taken into account, illumination effects can play an important role in achieving realism. Among illumination effects, shadows are often integral part of natural scenes and one of the elements contributing to naturalness of synthetic scenes. In this thesis, the problem of extracting shadows from digital images is discussed. A new analysis method for the segmentation of cast shadows in still and moving images without the need of human supervision is proposed. The problem of separating moving cast shadows from moving objects in image sequences is particularly relevant for an always wider range of applications, ranging from video analysis to video coding, and from video manipulation to interactive environments. Therefore, particular attention has been dedicated to the segmentation of shadows in video. The validity of the proposed approach is however also demonstrated through its application to the detection of cast shadows in still color images. Shadows are a difficult phenomenon to model. Their appearance changes with changes in the appearance of the surface they are cast upon. It is therefore important to exploit multiple constraints derived from the analysis of the spectral, geometric and temporal properties of shadows to develop effective techniques for their extraction. The proposed method combines an analysis of color information and of photometric invariant features to a spatio-temporal verification process. With regards to the use of color information for shadow analysis, a complete picture of the existing solutions is provided, which points out the fundamental assumptions, the adopted color models and the link with research problems such as computational color constancy and color invariance. The proposed spatial verification does not make any assumption about scene geometry nor about object shape. The temporal analysis is based on a novel shadow tracking technique. On the basis of the tracking results, a temporal reliability estimation of shadows is proposed which allows to discard shadows which do not present time coherence. The proposed approach is general and can be applied to a wide class of applications and input data. The proposed cast shadow segmentation method has been evaluated on a number of different video data representing indoor and outdoor real-world environments. The obtained results have confirmed the validity of the approach, in particular its ability to deal with different types of content and its robustness to different physically important independent variables, and have demonstrated the improvement with respect to the state of the art. Examples of application of the proposed shadow segmentation tool to the enhancement of video object segmentation, tracking and description operations, and to video composition, have demonstrated the advantages of a shadow-aware video processing.

Performance measures for quantifying human color constancy and computational color constancy are very different. The former relate to measurements on individual object colors whereas the latter relate to the accuracy of the estimated illuminant. To bridge this gap, we propose a psychophysical method in which observers judge the global color fidelity of the visual scene rendered under different illuminants. In each experimental trial, the scene is rendered under three illuminants, two chromatic test illuminants and one neutral reference illuminant. Observers indicate which of the two test illuminants leads to better color fidelity in comparison to the reference illuminant. Here we study multicolor scenes with chromatic distributions that are differently oriented in color space, while having the same average chromaticity. We show that when these distributions are rendered under colored illumination they lead to different perceptual estimates of the color fidelity.

Computational color constancy is the task of estimating the true reflectances of visible surfaces in an image. In this paper we follow a line of research that assumes uniform illumination of a scene, and that the principal step in estimating reflectances is the estimation of the scene illuminant. We review recent approaches to illuminant estimation, firstly those based on formulae for normalisation of the reflectance distribution in an image - so-called grey-world algorithms, and those based on a Bayesian formulation of image formation. In evaluating these previous approaches we introduce a new tool in the form of a database of 568 high-quality, indoor and outdoor images, accurately labelled with illuminant, and preserved in their raw form, free of correction or normalisation. This has enabled us to establish several properties experimentally. Firstly automatic selection of grey-world algorithms according to image properties is not nearly so effective as has been thought. Secondly, it is shown that Bayesian illuminant estimation is significantly improved by the improved accuracy of priors for illuminant and reflectance that are obtained from the new dataset.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

We investigated several sensory and cognitive determinants of colour constancy across 40 illumination hues. In the first experiment, we measured colour naming for the illumination and for the colour induced by the illumination on the colorimetric grey. Results confirmed that the induced colours are approximately complementary to the colour of the illumination. In the second experiment, we measured colour constancy using achromatic adjustments. Average colour constancy was perfect under the blue daylight illumination and decreased in colour directions away from the blue daylight illumination due to undershooting and a strong blue bias. Apart from this blue bias, colour constancy was not related to illumination discrimination and to chromatic detection measured previously with the same setup and stimuli. We also observed a strong negative relationship between the degree of colour constancy and the consensus of naming the illumination colour. Constancy coincided with a low naming consensus, in particular because bluish illumination colours were sometimes seen as achromatic. Blue bias and category consensus alone explained >68%, and all determinants together explained >94% of the variance of achromatic adjustments. These findings suggest that colour constancy is optimised for blue daylight.

Using convolutional neural network models illumination estimation according to light colors

Color constancy denotes the phenomenon that the appearance of an object remains fairly stable under changes in illumination and background color. Most of what we know about color constancy comes from experiments using flat, matte surfaces placed on a single plane under diffuse illumination simulated on a computer monitor. Here we investigate whether material properties (glossiness and roughness) have an effect on color constancy for real objects. Subjects matched the color and brightness of cylinders (painted red, green, or blue) illuminated by simulated daylight (D65) or by a reddish light with a Munsell color book illuminated by a tungsten lamp. The cylinders were either glossy or matte and either smooth or rough. The object was placed in front of a black background or a colored checkerboard. We found that color constancy was significantly higher for the glossy objects compared to the matte objects, and higher for the smooth objects compared to the rough objects. This was independent of the background. We conclude that material properties like glossiness and roughness can have significant effects on color constancy.

This paper addresses color constancy, the problem of finding true color of objects independent of the light illuminating the scene. However, many algorithms exist in this scope, they are all based on specific assumptions and none of them is universal. Therefore, in order to achieve better performance, some new color constancy methods have been proposed, which most of them are combinational algorithms. In this paper, a new combinational method is proposed; the proposed method consists of two steps: first, a classifier is used to determine the best group of color constancy algorithms for the input image; then, some of the algorithms in this group are combined to estimate the scene illuminant. In this way, it always combines the algorithms that have good performance for the input image, and as a result, the overall performance increases. The proposed method has been evaluated using multiple benchmark datasets, and the experimental results showed that the proposed approach outperformed current state-of-the-art algorithms.

Color constancy refers to the ability of the human visual system to stabilize the color appearance of surfaces under an illuminant change. In this work we studied how the interrelations among nine colors are perceived under illuminant changes, particularly whether they remain stable across 10 different conditions (5 illuminants and 2 backgrounds). To do so we have used a paradigm that measures several colors under an immersive state of adaptation. From our measures we defined a perceptual structure descriptor that is up to 87% stable over all conditions, suggesting that color category features could be used to predict color constancy. This is in agreement with previous results on the stability of border categories [1,2] and with computational color constancy algorithms [3] for estimating the scene illuminant.

The color constancy is a procedure that measures the influence of different light sources on a digital image. The goal of the computational color constancy is to account for the effect of the illuminate.Many traditional methods such as Grey-world method, Max RGB and learning-based method were used to measure the color constancy of digital images affected by light source. All these methods have an obvious disadvantage that the light source across the scene is spectrally uniform. This assumption is often violated as there might be more than one light source illuminating the scene. This paper has proposed a modified edge based color constancy using non local means and the adaptive histogram equalization. The proposed algorithm has ability to reduce the uneven illuminate problem of edge based color constancy and also non-local means has produced sharper results. The proposed algorithm is designed and implemented in the MATLAB using image processing toolbox. The comparison among the modified edge based color constancy and the edge based color constancy has shown that the proposed algorithm is quite effective over the available methods.