Robust histogram construction from color invariants

Color is a very important cue for object recognition, which can help increase the discriminative power of an object-recognition system and also make it more robust to variations in the lighting and imaging conditions. Nonetheless, even though most image acquisition devices provide color data, a lot of object-recognition systems rely solely on simple grayscale information. Part of the reason for this is that although color has advantages, it also introduces some complexities. In particular, the RGB values of a digital color image are only indirectly related to the surface “color” of an object, which depends not only on the object’s surface reflectance but also on such factors as the spectrum of the incident illumination, surface gloss, and the viewing angle. As a result, there has been a great deal of research into color invariants that encode color information but at the same time are insensitive to these other factors. This chapter describes these color invariants, their derivation, and their application to color-based object recognition in detail. Recognizing objects using a simple global image matching strategy is generally not very effective since usually an image will contain multiple objects, involve occlusions, or be captured from a different viewpoint or under different lighting conditions than the model image. As a result, most object-recognition systems describe the image content in terms of a set of local descriptors—SIFT, for example—that describe the regions around a set of detected keypoints. This chapter includes a discussion of the three color-related choices that need to be made when designing an object-recognition system for a particular application: Color-invariance, keypoint detection, and local description. Different object-recognition situations call for different classes of color invariants depending on the particular surface reflectance and lighting conditions that will be encountered. The choice of color invariants is important because there is a trade-off between invariance and discriminative power. All unnecessary invariance is likely to decrease the discriminative power of the system. Consequently, one part of this chapter describes the assumptions underlying the various color invariants, the invariants themselves, and their invariance properties. Then with these color invariants in hand, we turn to the ways in which they can be exploited to find more salient keypoints and to provide richer local region descriptors. Generally but not universally, color has been shown to improve the recognition rate of most object-recognition systems. One reason color improves the performance is that including it in keypoint detection increases the likelihood that the region surrounding the keypoint will contain useful information, so descriptors built around these keypoints tend to be more discriminative. Another reason is that color-invariant-based keypoint detection is more robust to variations in the illumination than grayscale-based keypoint detection. Yet another reason is that local region descriptors based on color invariants more richly characterize the regions, and are more stable relative to the imaging conditions, than their grayscale counterparts.

In recent years, kernel density estimation has been exploited by computer scientists to model several important problems in machine learning, bioinformatics, and computer vision. However, in case the dimension of the data set is high, then the conventional kernel density estimators suffer poor convergence rates of the pointwise mean square error (MSE) and the integrated mean square error (IMSE). Therefore, design of a novel kernel density estimator that overcomes this problem has been a great challenge for many years. This paper proposes a relaxed model of the variable kernel density estimation and analyzes its performance in data classification applications. It is proved in this paper that, in terms of pointwise MSE, the convergence rate of the relaxed variable kernel density estimator can approach O(n/sup -1/) regardless of the dimension of the data set, where n is the number of sampling instances. Experiments with the data classification applications have shown that the improved convergence rate of the pointwise MSE leads to higher prediction accuracy. In fact, the experimental results have also shown that the data classifier constructed based on the relaxed variable kernel density estimator is capable of delivering the same level of prediction accuracy as the SVM with the Gaussian kernel.

Maneuvering targets tracking in cluttered environment is a challenging problem in computer vision because of the difficulty of distinguishing the target from the background. In this paper, we treat tracking as a binary classification problem and employ support vector machine to suppress the background. In order to enhance the robustness against illumination changes, we propose to combine color invariance with traditional RGB values to train the SVM. First, we use expectation maximization algorithm to extract the target from the environment; then, RGB and color invariance values are used to train SVM. In the incoming frames, pixels in regions of interest are classified by SVM and the confidence map is produced, which will afterward be used by traditional tracking approach to track the target, in this paper, we employ particle filter. Experimental results on challenging sequences validate the effectiveness of the proposed method in cluttered background target tracking.

Color-based object recognition

Most state-of-the-art approaches to object and face detection rely on intensity information and ignore color information, as it usually exhibits variations due to illumination changes and shadows, and due to the lower spatial resolution in color channels than in the intensity image. We propose a new color descriptor, derived from a variant of Local Binary Patterns, designed to achieve invariance to monotonic changes in chroma. The descriptor is produced by histograms of encoded color texture similarity measures of small radially-distributed patches. As it is based on similarities of local patches, we expect the descriptor to exhibit a high degree of invariance to local appearance and pose changes. We demonstrate empirically by simulation the invariance of the descriptor to photometric variations, i.e. illumination changes and image noise, geometric variations, i.e. face pose and camera viewpoint, and discriminative power in a face detection setting. Lastly, we show that the contribution of the presented descriptor to face detection performance is significant and superior to several other color descriptors, which are in use for object detection. This color descriptor can be applied in color-based object detection and recognition tasks.

This paper presents a new approach for automatic image color correction, based on statistical learning. The method both parameterizes color independently of illumination and corrects color for changes of illumination. This is useful in many image processing applications, such as image segmentation or background subtraction. The motivation for using a learning approach is to deal with changes of lighting typical of indoor environments such as home and office. The method is based on learning color invariants using a modified multi‐layer perceptron (MLP). The MLP is odd‐layered. The middle layer includes two neurons which estimate two color invariants and one input neuron which takes in the luminance desired in output of the MLP. The advantage of the modified MLP over a classical MLP is better performance and the estimation of invariants to illumination. The trained modified MLP can be applied using look‐up tables, yielding very fast processing. Results illustrate the approach and compare it with other color correction approaches from the literature. © 2007 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 17, 132–142, 2007

Color histograms are widely used for content-based image retrieval because they are trivial and effective to compute, not sensitive to size and orientation, and robustly tolerate movement of objects in the image and changes in camera viewpoint. However, a color histogram only records an image's overall color compositions and no any spatial information is included, and also retrieval precision is limited. In this paper, a color-spatial histogram, which incorporates spatial information with color compositions without sacrificing the robustness of color histograms, is proposed. The purpose is considering the representation role of color compositions and spatial information in the meantime. Each entry in a color-spatial histogram is the color frequency and corresponding position information of the virtual boundary. Experimental evidence suggests that this new color-spatial histogram outperform not only the traditional color histogram method but also the other color-spatial histogram methods for image retrieval.

Cascade of descriptors to detect and track objects across any network of cameras

This paper aims for color constant identification of object colors through the analysis of spectral color data. New computational color models are proposed which are not only invariant to illumination variations (color constancy) but also robust to a change in viewpoint and object geometry (color invariance). Color constancy and invariance is achieved by spectral imaging using a white reference, and based on color ratio’s (without a white reference). From the theoretical and experimental results it is concluded that the proposed computational methods for color constancy and invariance are highly robust to a change in SPD of the light source as well as a change in the pose of the object.

With the development of the internet, more and more images appear in the internet. How to effectively retrieve the desired image is still an important problem. In the past, traditional color histogram is used image retrieval system, but color histograms lack spatial information and are sensitive to intensity variation, color distortion and cropping. As a result, images with similar histograms may have totally different semantics. So the spatial information should be included in color histogram. The color histogram based on saliency map approach is introduced to overcome the above limitations. In this paper, we present a robust image retrieval based on color histogram of saliency map. Firstly, in order to extract useful spatial information of each pixel, the steady saliency map of the images is extracted. Then, color histogram based on saliency map is introduced, and the similarity between color images is computed by using the color histogram of saliency map. Experimental results show that the proposed color image retrieval is more accurate and efficient in retrieving the user-interested images.

The problem of object or scene recognition is often addressed by seeking geometric image properties that are invariant under changes in viewing conditions. An alternative, non-geometric, ratio method was described by Funt and Finlayson (IEEE Trans. Pattern Anal. Mach. Intell. 17,522, 1995) in which histograms of spatial ratios of colour RGB triplets from neighbouring image regions were used to recognise objects under changes in viewpoint and illumination. In this study, ratio indexing was extended from RGB images to hyperspectral images with a variable number of sensor channels distributed over 400-720 nm. Fifty natural scenes were used to generate test and reference images. For each number of sensors, independent random samples were drawn from each test image of a scene under either a daylight of correlated colour temperature of 25000 K or of 4000 K and matched against independent random samples drawn from each reference image of the scenes under a daylight of correlated colour temperature 6500 K. Matching was based on the intersection of multi-dimensional histograms of ratios of sensor signals in these samples Differences between match hit and false-alarm rates provided a measure of recognition performance. Results suggest that for small samples, indexing with five sensor channels has advantages over indexing with three sensor channels for the recognition of natural scenes.

A conventional color histogram (CCH) considers neither the color similarity across different bins nor the color dissimilarity in the same bin. Therefore, it is sensitive to noisy interference such as illumination changes and quantization errors. Furthermore, CCHs large dimension or histogram bins requires large computation on histogram comparison. To address these concerns, this paper presents a new color histogram representation, called fuzzy color histogram (FCH), by considering the color similarity of each pixel's color associated to all the histogram bins through fuzzy-set membership function. A novel and fast approach for computing the membership values based on fuzzy c-means algorithm is introduced. The proposed FCH is further exploited in the application of image indexing and retrieval. Experimental results clearly show that FCH yields better retrieval results than CCH. Such computing methodology is fairly desirable for image retrieval over large image databases.

Background subtraction is a very popular approach for vehicle detection in traffic surveillance systems. A conventional color histogram (CCH) considers neither the color similarity across different bins nor the color dissimilarity in the same bin. Therefore, it is sensitive to noisy interference such as illumination changes and quantization errors. Furthermore, CCHs large dimension or histogram bins requires large computation on histogram comparison. However, structured motion patterns of the background (e.g., Vehicular traffic videos, waving leaves, spouting fountain, rippling water, etc.), which are distinctive from variations due to noise, are hardly tolerated in this assumption. To address these concerns, we introduce a background subtraction algorithm for temporally dynamic texture scenes. Specifically, we propose to adopt a clustering-based feature, called fuzzy color histogram (FCH), which has an ability of greatly attenuating color variations generated by background motions while still highlighting moving objects. Experimental results demonstrate that the proposed method is effective for background subtraction in dynamic texture scenes using LFCH features with adaptive updating procedure compared to several competitive methods proposed in the literature.

A new tracker based on particle filter is proposed in this paper. In our framework, colour cue and edge cue, which are represented as colour histogram (CH) and improved histogram of oriented gradient (IHOG) respectively, are adaptively fused to represent the target observation. Colour histogram is robust to shape variation and rotation etc, but sensitive to varying illumination and easy to be confused by distractions from background due to loss of spatial information; whereas for IHOG, the situation is reversed. With the help of the complementary nature of the two kinds of image features, the proposed tracker is more robust to pose variations, illumination changes and distractions from background. As the second contribution of this paper, an improved model update scheme is proposed to address the varying appearance. The new scheme makes our object model has a better resistance to template drift. Experimental results demonstrate the high robustness and effectiveness of our method in complex environments.

There are many research efforts in object recognition. Most existing methods for object recognition are based on full objects. However, many images contain multiple objects with occluded shapes and regions. Due to the occlusion of objects, image retrieval can provide incomplete, uncertain, and inaccurate results. To resolve this problem, we propose a new method to reconstruct objects using symmetry properties since most objects in a given image database are represented by symmetrical figures. Even though there have been several efforts in object recognition with occlusion, current methods have been highly sensitive to object pose, rotation, scaling, and visible portion of occluded objects. In addition, many appearance-based and model-based object recognition methods assumed that they have known occluded regions of objects or images through extensive training processes with statistical approach. However, our new approach is not limited to recognizing occluded objects by pose and scale changes, and does not need extensive training processes. Unlike existing methods, the proposed method finds shapes and regions to reconstruct occluded shapes and regions within objects. We assume that we only consider the elliptical objects in recognition. The proposed approach can handle object rotation and scaling for dealing with occlusion, and does not require extensive training processes. The main advantage of our proposed approach is that it becomes simple to reconstruct objects from occlusions using symmetry. We present a robust method, which is based on the contours of objects, for recognizing partially occluded objects based on symmetry properties. The contour-based approach finds a symmetry axis using the maximum diameter from the occluded object. In experiments, we demonstrate how a proposed method reconstructs and recognizes occluded shapes and regions using symmetry. Experiments use rotated and scaled objects for dealing with occlusion. We use mirror symmetry to find possible occluded regions in objects. Examples of partially occluded objects are shown in Figure 1.1. We also evaluate the recognition rate of the reconstructed objects using symmetry and the visible portion of the occluded objects for recognition. The method produces average recognition rates for cups and plates above 88% with 30% occlusion. In this case, part of the