Morphologically Decoupled Structured Sparsity for Rotation-Invariant Hyperspectral Image Analysis

Hyperspectral image processing is a very rapidly growing technology these days. It extends the scope of its applications across diversified fields such as Remote sensing, precision agriculture, medicine, Computer Vision applications and many more. Especially Hyperspectral Imagery, benefiting from its very own nature of providing rich spectral information, find many apllications in the field of Remote sensing and Earth observation. With the development of hyperspectral imaging sensor technology, the supply of HSI data has increased greatly, creating a need for proportionate rise in the HIS Processing techniques for better interpretation of data. In this context Deep Learning has become a viable option to handle huge datasets like Hyperspectral Imagery. Thus many enthusiastic researchers are showing interest in the domain of Hyperspectral Image Processing. This paper attempts to provide the basic concepts of HSI acquisition and processing at an abstract level with more focus given to Deep Learning based HSI processing tasks. So that the work will come in handy to the beginners in this field to get the preliminary concepts of HSI at a glance. To the best of our knowledge this work is first of its kind to bring to together in this study the the concepts of HSI acquisition and HSI processing.

Non-overlapping classification of hyperspectral imagery with superpixel segmentation

Combining spectral and spatial information can significantly improve the classification performance of hyperspectral image (HSI). Currently, a lot of spectral–spatial HSI classification methods have been proposed. However, the task of HSI classification has remained challenging since the number of training samples is limited in real scenarios. In this article, we propose a novel HSI classification framework with single sample, in which the spectral self-similarity and spatial polygon structure information are fully combined to improve the classification performance. On the one hand, spectral self-similarity is used to expand training samples, which makes it possible to obtain sufficient samples with minimal cost. On the other hand, polygonal partition is introduced to acquire the geometrical structure of land covers in man-made environments. Specifically, the edge information of geometric objects is captured by polygonal partition, which can be utilized to constrain the spatial range of sample expansion and optimize the classification results. Experimental results on three real HSIs illustrate that the proposed method performs very well under small training sample size even when the number of samples is single per class.

Dimensionality reduction is a key step in hyperspectral image processing. Recent investigations are looking into nonlinear manifold learning for dimensionality reduction in hyperspectral imagery. Nonlinear manifold learning methods such as Isomap, and Local Linear Embedding are use to recover the low dimensional representation of an unknown nonlinear manifold for high dimensional data where it is important to retain the neighborhood structure of the manifold. Although these algorithms use different philosophies for recovering the nonlinear manifold, they all incorporate neighborhood information from each data point to construct a weighted graph having the data points as vertices. Thus the performance of these methods highly depends on how well these neighborhoods are selected, since all subsequent steps rely on it. The k -NN algorithm is the most widely used technique for neighbor selection in manifold learning. However, it can result in a disconnected graph and it does not fully exploit spatial neighborhood information from the image in selecting points to form the neighborhoods. In this paper, recently proposed methods for constructing the weighted graph in manifold learning are studied: k -VC, k -EC and k -MST, which have the advantage of creating connected graphs and have performed well in artificial data sets. Spatial information of the hyperspectral images is included in the manifold learning process by using spatial coherence. Experiments are conducted with artificial data and hyperspectral images. For hyperspectral images, classification accuracy was used as an indirect measure to understand how the low dimensional embedding of the data reduces dimensionality while still maintaining good discrimination performance.

Target detection is a key issue in processing hyperspectral images (HSIs). Spectral-identification-based algorithms are sensitive to spectral variability and noise in acquisition. In most cases, both the target spatial distributions and the spectral signatures are unknown, so each pixel is separately tested and appears as a target when it significantly differs from the background. In this paper, we propose two algorithms to improve the signal-to-noise ratio (SNR) of hyperspectral data, leading to detectors that are robust to noise. These algorithms consist in integrating adaptive spatial/spectral filtering into the adaptive matched filter and adaptive coherence estimator. Considering the HSIs as tensor data, our approach introduces a data representation involving multidimensional processing. It combines the advantages of spatial and spectral information using an alternating least square algorithm. To estimate the signal subspace dimension in each spatial mode, we extend the Akaike information criterion, and we develop an iterative algorithm for spectral-mode rank estimation. We demonstrate the interest of integrating the quadtree decomposition to perform an adaptive 3-D filtering and thereby preserve the local image characteristics. This leads to a significant improvement in terms of denoised tensor SNR and, consequently, in terms of detection probability. The performance of our method is exemplified using simulated and real-world HYperspectral Digital Imagery Collection Experiment images.

The intent of this study was to develop a technique for weight and ripeness estimation of oil palm (Elaeis guieensis Jacq. var. tenera) bunches from hyperspectral and RGB color images. In the experiments, color and hyperspectral images of the bunch were acquired from four different angles, each differing by 90 degrees. Acquired RGB color images were converted to HSI and L*a*b color space. Gray-scale thresholds were used to identify the area of the bunch and the area of space between the fruits. The total number of pixels in the bunch and the space were counted, respectively. In the hyperspectral images, the total number of pixels in the bunch was also counted from an image composed of three wavelengths (560 nm, 680 nm, and 740 nm), while the total number of pixels of space between fruits was obtained at a wavelength of 910 nm. From these sets of data, weight-estimation equations were determined by linear regression (LR) or multiple linear regression (MLR). As a result, the coefficient of determination (R2) of actual weight and estimated weight were at a level of 0.989 and 0.992 for color and hyperspectral images, respectively. Estimation of oil palm bunch ripeness was also tested. Bunches belonging to 4 classes of ripeness (overripe, ripe, underripe, and unripe) were used for this study. Since ripeness estimation from overall data from a bunch was quite difficult, we focused on the difference in colors or reflectivity of the portion concealed and not-concealed with fronds. Euclidean distances between the test sample and the standard 4 classes of ripeness were calculated, and the test sample was classified into the ripeness class that had the shortest distance from the sample. In the classification based on color image, average RGB values of concealed and not-concealed areas were used, while in hyperspectral images the average intensity values of fruits pixels from the concealed area were used. The results of validation experiments with the developed estimation methods indicated acceptable estimation accuracy, and a possibility for practical use to estimate the ripeness of oil palm bunches.

Extended morphological profiles with reconstruction are widely used in the classification of very high resolution hyperspectral data from urban areas. However, morphological profiles constructed by morphological openings and closings with reconstruction can lead to some undesirable effects. Objects expected to disappear at a certain scale remain present when using morphological openings and closings by reconstruction. In this paper, we apply extended morphological profiles with partial reconstruction (EMPP) to the classification of high resolution hyperspectral images from urban areas. We first used feature extraction to reduce the dimensionality of the hyperspectral data, as well as reduce the redundancy within the bands, then constructed EMPP on features extracted by PCA, independent component analysis and kernel PCA for the classification of high resolution hyperspectral images from urban areas. Experimental results on real urban hyperspectral image demonstrate that the proposed EMPP built on kernel principal components gets the best results, particularly in the case with small training sample sizes.

This work addresses the problem of hyperspectral image classification when the number of labeled samples is very small (few shot learning). Our work is based on the recently proposed framework of convolutional transform learning. In this work, we propose a semi-supervised version of deep convolutional transform learning. We compare with four recent studies which are tailored for solving the few-shot learning problem in hyperspectral classification. Results show that our method can improve over the state-of-the-art.

Analysis of hyperspectral images is of great importance in many scientific disciplines. Obtaining the spectral and spatial information simultaneously from time-varying hyperspectral images is a challenging task due to their high dimensionality. In this paper, we design an interface that allows users to study hyperspectral images interactively and obtain spectral features and enhanced images at the same time. The image fusion results change dynamically with the regions of interest selected by users and convey both the spatial and spectral information. We show the usefulness of our approach using time-varying hyperspectral plant images. We compare our method with existing hyperspectral image analysis techniques. Our evaluation indicates that our interface can help users determine important bands, identify regions of interest, and generate image fusion results for time-varying hyperspectral plant images.

Deep learning methods have been successfully applied for multispectral and hyperspectral images classification due to their ability to extract hierarchical abstract features. However, the performance of these methods relies heavily on large-scale training samples. In this paper, we propose a three-dimensional spatial-adaptive Siamese residual network (3D-SaSiResNet) that requires fewer samples and still enhances the performance. The proposed method consists of two main steps: construction of 3D spatial-adaptive patches and Siamese residual network for multiband images classification. In the first step, the spectral dimension of the original multiband images is reduced by a stacked autoencoder and superpixels of each band are obtained by the simple linear iterative clustering (SLIC) method. Superpixels of the original multiband image can be finally generated by majority voting. Subsequently, the 3D spatial-adaptive patch of each pixel is extracted from the original multiband image by reference to the previously generated superpixels. In the second step, a Siamese network composed of two 3D residual networks is designed to extract discriminative features for classification and we train the 3D-SaSiResNet by pairwise inputting the training samples into the networks. The testing samples are then fed into the trained 3D-SaSiResNet and the learned features of the testing samples are classified by the nearest neighbor classifier. Experimental results on three multiband image datasets show the feasibility of the proposed method in enhancing classification performance even with limited training samples.

Recently, the sparse representation based classification methods have received particular attention in the classification of hyperspectral imagery. However, current sparse representation based classification models have not considered all the test pixels simultaneously. In this paper, we propose a hyperspectral classification method with spatial filtering and norm (SFL) that can deal with all the test pixels simultaneously. The norm regularization is used to extract relevant training samples among the whole training data set with joint sparsity. In addition, the norm loss function is adopted to make it robust for samples that deviate significantly from the rest of the samples. Moreover, to take the spatial information into consideration, a spatial filtering step is implemented where all the training and testing samples are spatially averaged with its nearest neighbors. Furthermore, the non-negative constraint is added to the sparse representation matrix motivated by hyperspectral unmixing. Finally, the alternating direction method of multipliers is used to solve SFL. Experiments on real hyperspectral images demonstrate that the proposed SFL method can obtain better classification performance than some other popular classifiers.

Surface water is an essential carrier for the balance of ecosystems and human existence. As applied hyperspectral remote sensing has improved, the research into water extraction from the hyperspectral image gradually attracted more attention. However, delineating water from the hyperspectral image is challenging due to shadows of buildings and trees, and other dark pixels. Furthermore, water indexes that come from multispectral imagery are directly applied to hyperspectral imagery leads to some defects, such as underutilized abundant information and higher misclassifications. Here, we proposed a decision tree water index (DTWI) method to extract water bodies from hyperspectral imagery. Combining the threshold of the near-infrared reflectance and the magnitude of the difference between reflectance of 700 and 730 nm, the DTWI method could effectively extract water from hyperspectral imagery. Three hyperspectral images derived from the study areas within the city, a mixture of city and suburban zone, and suburban zone only, were applied to test the feasibility and flexibility of this method in various sensors and environments. Four alternative methods were used to verify the robustness of the proposed method, including hyperspectral difference water index, normalized difference water index, shaded building index, and support vector machine (SVM). Our results indicated that the proposed method was better suited to extracting water bodies from hyperspectral imagery when compared with other methods and could effectively capture water on different hyperspectral sensors, various heterogeneous surroundings, and multi-spatial scales. The proposed DTWI method has high stability in threshold and band selection, which is comparable to the SVM method in the mean overall accuracy and kappa coefficient (DTWI: 0.98 and 0.81; SVM: 0.98 and 0.82). For the view of elapsed time, calculation, and applicability, the proposed method has the potential of better utilization and rapid response for water extraction in hyperspectral imagery.

Acquired images in hyperspectral imagery are disturbed by additive noise which is usually assumes as zero-mean white one. In fact, there is still non-white noise in hyperspectral images (HSIs). The 2-dimensional filtering methods and multidimensional tensor decomposition algorithms cannot be used to remove non-white noise from HSIs directly. Therefore, a prewhitening denoising solution based on tensor decomposition is proposed in this paper. The prewhitening procedure can help to transform the non-white noise into a white one and the tensor decomposition method removes the noise without affecting the related information between image planes. The simulation results show that this non-white noise reducing approach has promising prospects in this field compared to existing denoising methods.

Despite the successful applications of unsupervised sparse dimensionality reduction (USDR) in pattern recognition, the USDR still suffers from two challenges for hyperspectral images (HSIs), which limit its discriminative performance: first, it cannot be applied for dimensionality reduction using both training samples and testing samples; second, it lacks the ability to integrate the spectral with spatial information for improving the discriminative performance of HSIs. In order to tackle the first challenge, we extend it to a supervised scenario, which can be applied for both training samples and testing samples, namely dimensionality reduction sparse representation (DRSR). Then, we propose a novel method called local and global DRSR (LGDRSR) to integrate the spectral information and spatial information of HSIs to further improve the discriminative performance of HSIs. The proposed LGDRSR computes the distance information between pixels of HSIs including the whole samples and their corresponding locations with a unified metric matrix. Experimental results show the proposed LGDRSR outperforms other state-of-the-art algorithms significantly.

Algorithms and information technologies for processing Earth hyperspectral imagery are presented. Several new approaches are discussed. Peculiar properties of processing the hyperspectral imagery, such as multifold signal-to-noise reduction, atmospheric distortions, access to spectral characteristics of every image point, and high dimensionality of data, were studied. Different measures of similarity between individual hyperspectral image points and the effect of additive uncorrelated noise on these measures were analyzed. It was shown that these measures are substantially affected by noise, and a new measure free of this disadvantage was proposed. The problem of detecting the observed scene object boundaries, based on comparing the spectral characteristics of image points, is considered. It was shown that contours are processed much better when spectral characteristics are used instead of energy brightness. A statistical approach to the correction of atmospheric distortions, which makes it possible to solve the stated problem based on analysis of a distorted image in contrast to analytical multiparametric models, was proposed. Several algorithms used to integrate spectral zonal images with data from other survey systems, which make it possible to image observed scene objects with a higher quality, are considered. Quality characteristics of hyperspectral data processing were proposed and studied.