Multiple 3-D Feature Fusion Framework for Hyperspectral Image Classification

Compressive sensing (CS) has been employed to compress and store hyperspectral images (HSI) to transfer extensive data efficiently, and obtaining high-quality reconstructed images is crucial for subsequent applications. Model-based methods usually capture the priors of HSI, including sparsity, global and nonlocal low-rankness, nonlocal self-similarity and global correlation. However, it is challenging to reasonably integrate different regularizations in a united framework and exploit various priors mentioned to reconstruct high-quality images. In this paper, a tensor optimization model based on the low-rank coefficient tensor and global prior (LRCTGP) is proposed for HSI reconstruction. Complementary regularization terms are integrated into a united framework and effectively promote the reconstruction results. First, we apply the mode- tensor-matrix product to decompose the original HSI rather than employing the regularization to capture the spectral low-rankness and simultaneously constrain the original and feature HSI. Then, tensor ring decomposition is employed to constrain the coefficient tensor with fewer bands, which is more efficient in the low-rank approximation and has smaller calculation costs than applying it to the original HSI. Moreover, integrating BM3D as a regularizer is more efficient than other patch-based models. Finally, considering the smoothness and global correlation, spatial-spectral total variation (SSTV) is applied to compensate for the shortcomings after decomposing the original and feature HSI, which complements the overall structure and details of reconstructed images and improves the reconstruction quality. Alternating direction method of multipliers (ADMM) is used to optimize the proposed model. Experimental results of the LRCTGP model on different HSI datasets are better than existing state-of-the-art approaches, which proves the superiority.

A spatial-spectral metric learning (SSML) framework for hyperspectral image (HSI) classification is proposed. SSML learns a metric by considering both the spectral characteristics and spatial features represented as the mean of neighboring pixels. It first performs the local pixel neighborhood preserving embedding (LPNPE) to reduce the dimensionality of HSI and meanwhile to preserve the spatial local similarity structure. Then, it learns a spectral and spatial distance metric, separately. Finally, the combination of the spectral and spatial metrics yields a joint spatial-spectral metric. It is followed by a nearest neighbor (NN) classifier for HSI classification. SSML shows good performance over the spectral and spatial NN and SVM on the benchmark hyperspectral data set of Indian Pines.

To build small and efficient neural networks for hyperspectral image (HSI) classification, this letter presents a depthwise separable residual neural network (ResNet). This approach, motivated by the popular MobileNet architecture, decomposes the traditional spatial-spectral convolution operation into a spatial-independent pointwise spectral convolution and a spectral-independent depthwise spatial convolution. It allows the separation of spectral and spatial information in HSI and also greatly reduces the network size to prevent the overfitting issue. To better preserve the class boundaries and edges, the proposed ResNet is integrated into a maximum a posteriori (MAP) framework to allow the use of the conditional random field (CRF) model. The experiment results on benchmark HSI scene demonstrate that the proposed ResNet compares favorably with several popular deep learning HSI classifiers and that the ResNet-CRF approach achieves higher accuracy and better boundaries among neighboring classes.

Hyperspectral image classification based on combining texture and multiscale guided filter feature of principal component

The Hyperspectral Image (HSI) is a great source of information for observing the earth’s elements due to its numerous narrow and continuous spectral wavelength bands. There are some key difficulties, such as the fact that the image bands are spectrally and spatially highly correlated, and the ‘curse of dimensionality’ in using the original HSI for classification. Band (dimensionality/feature) reduction is necessary to improve classification performance through feature extraction and selection. As such, this paper proposes a hybrid HSI classification paradigm, termed Hybrid-2DNET. Specifically, our proposed Hybrid-2DNET incorporates factor analysis-based feature extraction along the Minimum-Redundancy-Maximum-Relevance (mRMR)-based feature selection criterion and a 2D-wavelet Convolutional Neural Network (CNN) to reduce both spectral and spatial dimensionalities. The experimental analyses performed on three remote sensings HSI datasets, namely Indian Pines, University of Pavia, and Salinas Scene, manifest that our proposed Hybrid-2DNET outperforms the handcrafted and deep learning methods.

In this research, a deep learning approach for hyperspectral image (HSI) classification was developed, incorporating attention mechanisms, multiscale feature learning, and utilization of unsampled pixels. The proposed model, multiscale attention-based hybrid spectral network and UNet (MSA-HybridSN-U), was evaluated on three benchmark datasets: Indian Pines, University of Pavia, and University of Houston using Overall Accuracy (OA), Average Accuracy (AA), and Kappa index (K). The proposed model MSA-HybridSN-U had high accuracy on Indian Pines (IP), Pavia University (PU), and Houston University (HU) datasets. It achieved the highest average accuracy compared with other models like 3D CNN, HybridSN, M3D-DCNN, DBDA, and ACA-HybridSN and higher than 2D CNN and SVM. For the IP dataset, the OA and AA scores are both 99.71% and 99.65% respectively, indicating a high level of accuracy in classifying the samples. The Kappa statistic, which measures inter-annotator agreement, is also high at 0.997, suggesting that the method is consistent in its predictions. For the PU dataset, the results are even higher with OA and AA scores of 99.97% and 99.89% respectively, and a Kappa statistic of 0.999, indicating even higher accuracy and consistency. Finally, for the HU dataset, the results are similarly high with OA and AA scores of 99.47% and 99.92% respectively, and a Kappa statistic of 0.994. The use of attention mechanisms, multiscale features, and unsampled pixels has improved the classification performance of the model. The combination of spectral and spatial attention modules improved the accuracy the most, with the highest accuracy of 99.97% on the PU dataset. Utilizing unsampled pixels in the classification process resulted in a noticeable improvement in accuracy, particularly on the HU dataset with an improvement of 0.29%. The results show the effectiveness of combining multiscale features and attention modules in improving the accuracy of HSI classification.

The superpixel-based multiple kernels model uses the average value of all pixels within superpixel as the spatial feature, which results in inaccurate extraction of edge pixels. To solve this problem, a local binary patterns and superpixel-based multiple kernels method is proposed for hyperspectral image (HSI) classification. First, the original HSI is segmented into multiple superpixels by using the entropy rate superpixel segmentation algorithm. On the HSI with superpixel index, the spectral kernel is second obtained by combining the spectral feature map with the radial basis kernel (RBF). By introducing local binary pattern (LBP) and weighted average filtering into RBF, the spatial kernels are obtained within and among superpixels. Finally, the combined kernel containing the abovementioned three kernels is inputted into the support vector machine classifier to generate a classification map. The experimental procedure in this article uses LBP to extract the information in superpixels, which effectively prevents the loss of edge features in superpixels. The experimental results show that the proposed method is superior to the state-of-the-art classifiers for HSI classification.

Combining spectral and spatial information has been proven to be an effective way for hyperspectral image (HSI) classification. However, making full use of spectral-spatial information of HSI still remains an open problem, especially when only a small number of labeled samples are available. In this paper, a new spectral-spatial feature extraction method called locally homogeneous covariance matrix representation (LHCMR) is proposed for the fusion of spectral and spatial information. Specially, to make use of neighborhood homogeneity of land covers, original HSI is first segmented into many superpixels using our modified entropy rate superpixel segmentation. Then, to acquire the most similar pixels, we propose to construct neighborhoods of each pixel from the overlapping areas between the corresponding superpixels and the sliding window centered on it. Subsequently, covariance matrix representations (CMRs) of different pixels can be obtained. In the classification stage, we fed the obtained CMRs into SVM with Log-Euclidean-based kernel for classification. Compared to the traditional approach that utilizes neighboring information only within a fixed window, the proposed local homogeneity strategy can absorb more discriminative spectral-spatial features. Experimental results from a series of available HSI datasets show that our proposed method is superior to several state-of-the-art methods, especially when the training set is very limited.

Local binary pattern (LBP) is a simple yet efficient image feature description algorithm that has been widely used in applications related to image classification. To more effectively make use of the local features of a hyperspectral image (HSI), a tensor-based spatial–spectral local binary pattern (TSSLBP) that can fully utilize the local spatial-spectral feature of an HSI is proposed. We specifically make efforts in three aspects. First, we construct a third-order tensor model for HSI to combine the local spatial and spectral features. Second, a dimension reduction method based on tensor decomposition is proposed to reduce redundant information of the HSI tensor model. Third, a joint spatial-spectral LBP coding algorithm is implemented based on the HSI tensor model to calculate the local spatial–spectral feature vector of HSI. To verify the effectiveness of the proposed TSSLBP, we apply it to the task of HSI classification and compare it with six existing algorithms. Experimental results on the three popular HSI datasets show that TSSLBP can fully utilize rich information in the local spatial–spectral domain and can significantly improve the classification accuracy of HSI, especially in the case of scarce training samples.

A combination method of stacked autoencoder and 3D deep residual network for hyperspectral image classification

Semisupervised learning (SSL) is often used when the number of labeled samples is very small compared to the number of unlabeled samples. It permits the exploitation of structure within unlabeled samples during the learning task. Like many other applications, remote sensing images suffer from the limited number of ground-truth samples and therefore semisupervised techniques may be used to overcome this limitation. In this paper, a semisupervised framework is proposed for classification of hyperspectral images with scarce labeled samples. Our method, which we call SSL-CC, utilizes fuzzy spectral clustering to label spatially neighboring samples. SSL-CC is implemented and tested on two benchmark hyperspectral datasets. Fuzzy clustering is compared to traditional crisp clustering (k-means) and the obtained results indicate that fuzzy clustering can significantly improve classification accuracy. SSL-CC achieves on average 60% improvement over a baseline SVM classifier.

In this paper, we propose a novel semi-supervised learning classification framework using box-based smooth ordering and multiple 1D-embedding-based interpolation (M1DEI) in [J. Wang, Semi-supervised learning using multiple one-dimensional embedding-based adaptive interpolation, Int. J. Wavelets Multiresolut. Inf. Process. 14(2) (2016) 11 pp.] for hyperspectral images. Due to the lack of labeled samples, conventional supervised approaches cannot generally perform efficient enough. On the other hand, obtaining labeled samples for hyperspectral image classification is difficult, expensive and time-consuming, while unlabeled samples are easily available. The proposed method can effectively overcome the lack of labeled samples by introducing new labeled samples from unlabeled samples in a label boosting framework. Furthermore, the proposed method uses spatial information from the pixels in the neighborhood of the current pixel to better catch the features of hyperspectral image. The proposed idea is that, first, we extract the box (cube data) of each pixel from its neighborhood, then apply multiple 1D interpolation to construct the classifier. Experimental results on three hyperspectral data sets demonstrate that the proposed method is efficient, and outperforms recent popular semi-supervised methods in terms of accuracies.

Hyperspectral image classification using multi-feature fusion

Recently, graph convolutional network (GCN) has drawn increasing attention in hyperspectral image (HSI) classification, as it can process arbitrary non-Euclidean data. However, dynamic GCN that refines the graph heavily relies on the graph embedding in the previous layer, which will result in performance degradation when the embedding contains noise. In this letter, we propose a novel dual residual graph convolutional network (DRGCN) for HSI classification that integrates two adjacency matrices of dual GCN. In detail, one GCN applies a soft adjacency matrix to extract spatial features, the other utilizes the dynamic adjacency matrix to extract global context-aware features. Subsequently, the features extracted by dual GCN are fused to make full use of the complementary and correlated information among two graph representations. Moreover, we introduce residual learning to optimize graph convolutional layers during the training process, to alleviate the over-smoothing problem. The advantage of dual GCN is that it can extract robust and discriminative features from HSI. Extensive experiments on four HSI data sets, including Indian Pines, Pavia University, Salinas, and Houston University, demonstrate the effectiveness and superiority of our proposed DRGCN, even with small-sized training data.

Dual selective fusion transformer network for hyperspectral image classification.