Using a spectral reflectance model for the illumination-invariant recognition of local image structure

In this article, new local matching measures, based on fuzzy similarity, for stereo matching of color images are proposed and evaluated. By formulating the individual similarity between a pair of pixels as a conjunction of the similarities of the respective color components, the field programmable gate array implementation problem becomes more computationally tractable, since the word-length of the numbers carrying the similarity information can be reduced compared to standard techniques (Sum of absolute differences and Sum of squared differences). It is also shown that combining information about color and local image structure (horizontal gradient component) in the matching measure and using a multi-scale measure is advantageous compared to using standard measures, in terms of percentage of correct matches and MSE for the erroneous matches. However, these improvements come at the prize of more complex implementations. Since the techniques are window-based, they share the typical drawbacks associated with other techniques of this type. This is expected, since the focus is on developing new local matching measure without addressing issues like post-processing of the resulting disparity maps. The techniques have been tested on two stereo color image pairs.

In this paper, we aim at the classification of the physical nature of local image structures in color images on the basis of geometrical and photometrical information. To this end, a framework is proposed to combine the local differential structure (i.e. geometrical information such as edges, corners, T-junctions etc) and color (i.e. photometrical information such as shadows, shading, illumination, highlights) in a multi-dimensional feature space. This framework is used to yield a proper classifier to classify salient image structures on the basis of their physical nature. The proposed framework is empirically verified on a set of images. From the theoretical and experimental results it is concluded that the proposed classification scheme successfully classifies local image structures robust to image translation and rotation, illumination intensity variations, and noise.

Different kinds of local image structures (such as homogeneous, edge-like and junction-like patches) can be distinguished by the intrinsic dimensionality of the local signals. Intrinsic dimensionality makes use of variance from a point and a line in spectral representation of the signal in order to classify it as homogeneous, edge-like or junction-like. The concept of intrinsic dimensionality has been mostly exercised using discrete formulations; however, recent work (; ) has introduced a continuous definition. The current study analyzes the distribution of local patches in natural images according to this continuous understanding of intrinsic dimensionality. This distribution reveals specific patterns than can be also associated to local image structures established in computer vision and which can be related to orientation and optic flow features. In particular, we link quantitative and qualitative properties of optic-flow error estimates to these patterns. In this way, we also introduce a new tool for better analysis of optic flow algorithms.

In this paper, we propose a novel weakly supervised semantic segmentation (WSSS) method that uses image tags as supervision to achieve joint pixel-level localization of the key local structure (KLS) and image-level classification of the aurora images captured by the ground-based optical all-sky imager. First, a patch-scale model (PSM) based on the small-scale structure of aurora is designed to identify the type-specific regions for each training image. Second, a region-scale model is trained with the identified type-specific regions to coarsely localize the KLS from multiple sizes of field of view, based on which the aurora image is classified. Finally, given the predicted image type, the PSM further refines the KLS in a pixel level. By localizing KLS from coarse to fine, the proposed method captures both overall shape with a bottom-up processing and local structure details of aurora in a top-down manner. Extensive experiments on the expert labeled data sets have demonstrated the efficacy of the proposed method in benchmarking with the state-of-the-art WSSS methods.

Using eigencolor normalization for illumination-invariant color object recognition

Local directional ZigZag pattern: A rotation invariant descriptor for texture classification

We present a new approach to volumetric scene reconstruction which can produce accurate models from turntable image sequences. Instead of an epipolar plane image (EPI) volume, we consider a function on the 4D spatiotemporal volume valued with the intensity back projection of the camera (time) to the particular voxel. Using an optical flow technique we compute the local orientation of this spatiotemporal image and decide on occupancy based on the relative orientation between the viewing ray of the voxel at the particular time and the local image structure. Our method does not require a background compensation like the silhouette-based methods and is comparable in performance with space carving.

Feature selection has attracted considerable attention due to the wide application of multi-label learning. However, previous methods do not fully consider the relationship between feature sets and label sets but devote attention to either of them. Furthermore, conventional multi-label learning utilizes logical labels to estimate relevance between feature sets and label sets so that the importance of corresponding labels cannot be well reflected. Additionally, numerous irrelevant and redundant labels degrade the classification performance of models. To this end, we propose a multi-label feature selection method named Robust multi-label Feature Selection with shared Label Enhanced (RLEFS). First, we obtain a robust label enhancement term by reconstructing labels from logical labels to numerical labels and imposing \(l_{2,1}\)-norm onto the label enhancement term. Second, RLEFS utilizes the robust label enhancement term to share the similar latent semantic structure between feature matrix and label matrix. Third, local structure is considered to ensure the consistency of label information during the feature selection process. Finally, we integrate the above terms into one joint learning framework, and then, a simple but effective optimization method with provable convergence is proposed to solve RLEFS. Experimental results demonstrate the classification superiority of RLEFS in comparison with seven state-of-the-art methods.

Phase division and recognition of crystal HRTEM images based on machine learning and deep learning

The image color recorded in a computer vision system depends on three factors: the physical content of the scene, the illumination on the scene, and the characteristics of the camera. The goal of computational color correction is to reset the intensities of the color channels in digital images, and express the scene colors canonically. In the color correction method based on spectral reflectance reproduction, the finite-dimensional linear-model is an effective way to condense spectral reflectance. Since there are only three channels for each pixel in digital images, color correction based on spectral reflectance reproduction using the finite-dimensional model with three dimensions is needed. In the definite gamut, the spectral reflectance curves are analyzed using the finite-dimensional model. It is shown that reflectance generated by three basis functions which is capturing 99.13% of the overall variance, provides good approximations to the measured spectral reflectance. Together with the measured spectral power distribution of the illumination and the estimated spectral responses of the camera, the quantum catches are calculated. Errors caused by approximation of the finite-dimensional model are relatively small, which shows that color correction based on the finite-dimensional linear-model has feasibility to some extent.

Tensor-based sparse representations of multi-phase medical images for classification of focal liver lesions

Superpixel conditional generation adversarial network for CMR artifact correction

An accurate estimation of vegetation Gross Primary Productivity (GPP), which is the amount of carbon taken up by vegetation through photosynthesis for a given time and area, is critical for understanding terrestrial-atmosphere CO2 exchange processes, ecosystem functioning, and as well as ecosystem responses and adaptations to climate change. Earliest studies, based on ground, airborne and satellite Sun-Induced chlorophyll Fluorescence (SIF) observations have recently revealed close relationships with GPP at different spatial and temporal scales and across different plant functional type (PFT). However, questions remain regarding whether there is a unique relationship between SIF and GPP across different sites and PFT and how can we improve GPP estimates using solely remotely sensed data. Using concurrent measurements of daily TROPOMI (TROPOspheric Monitoring Instrument) SIF (daily SIFd), daily MODIS Terra and Aqua spectral reflectance, and vegetation indices (VIs, notably NDVI (normalized difference vegetation index), NIRv (near-infrared reflectance of vegetation) and PRI (photochemical reflectance index)) and daily tower-based GPP across eight major different PFT, including mixed forests, deciduous broadleaf forests, croplands, evergreen broadleaf forests, evergreen needleleaf forests, grassland, open shrubland, and wetland, the strength of the linear relationships between tower-based GPP and SIFd at 40 ICOS (Integrated Carbon Observation Systems) flux sites was investigated, and the synergy between these variables to improve GPP estimates using a data-driven modelling approach was evaluated. The results revealed that the strength of the linear relationship between GPP and SIFd was strongly site-specific and PFT-dependent. Furthermore, the GLM (Generalized Linear Model) model, fitted between SIFd, GPP, site and vegetation type as categorical variables, further supported this site-and PFT-dependent relationship between GPP and SIFd. This study also showed that the spectral reflectance bands (RF-R), SIFd plus spectral reflectance (RF-SIF-R) models explained over 80 % of the seasonal and interannual variations in GPP, whereas the SIFd plus VIs (RF-SIF-VI) model reproduced only 75 % of the tower-based GPP variance. In addition, the relative importance results demonstrated that the spectral reflectance bands in the near-infrared, red and SIFd appeared as the most influential and dominant factors determining GPP predictions, indicating the importance of canopy structure, biochemical properties and vegetation functioning on GPP estimates. Overall, this study provides insights into understanding the strength of the relationships between GPP and SIF and the use of the spectral reflectance and SIFd to improve GPP across sites and PFT.

The project of developing an intelligent Chinese chess playing robot in natural environment is facing many challenges in which a robust illumination invariant recognition of the Chinese printed character in a chess piece is addressed in this paper. Instead of illumination sensitive image binarization which inevitably causes information loss and even tends to fail recognition, the proposed approach performs directly Gabor transform on a local subimage which includes Chinese character to obtain directional Gabor filters' responses which, to some extent, reduce effects of light changes on feature extraction. The responses are binarized by an adaptive thresholding before feature vector which is highly tolerant to illumination changes has been extracted from the binary images. Radial basis function neural network (RBFN) is used to distinguish characters as a classifier. A database consists of 2000 generated Chinese character images with varying intensity and Gaussian noise is used to verify the performance of binarized Gabor filters and Gabor filters respectively. Experiment results reveal that the proposed approach is superior to Gabor filters and a promising 95% recognition rate of characters in natural scenes is observed.

This paper presents a new robust and invisible blind watermarking approach of colour images for copyright protection in hybrid DWT-DCT-SVD domain. In the proposed method, firstly the luminance component (Y) of the cover image is decomposed up to one level of discrete wavelet transform (DWT) coefficients and the low frequency band (LL) is transformed by discrete cosine transform (DCT). Then several selected low and intermediate frequency DCT coefficients of each block are extracted to generate a feature matrix and singular value decomposition (SVD) transform is applied to the feature matrix. Finally the watermark information is embedded by modifying the singular values of the feature matrix. Experimental results demonstrate that the proposed approach outperforms some popular existing watermarking methods in robustness against median filtering, Gaussian filtering, salt and pepper noise, average filtering, Gaussian noise, histogram equalisation, and so on, especially in case of lossy JPEG compression in addition to good imperceptibility.