A deep learning framework for hyperspectral image classification with mixed frequency-domain – spatial domain feature representation

During recent years, convolutional neural network (CNN)-based methods have been widely applied to hyperspectral image (HSI) classification by mostly mining the spectral variabilities. However, the spatial consistency in HSI is rarely discussed except as an extra convolutional channel. Very recently, the development of pixel pair features (PPF) for HSI classification offers a new way of incorporating spatial information. In this paper, we first propose an improved PPF-style feature, the spatial pixel pair feature (SPPF), that better exploits both the spatial/contextual information and spectral information. On top of the new SPPF, we further propose a flexible multi-stream CNN-based classification framework that is compatible with multiple in-stream sub-network designs. The proposed SPPF is different from the original PPF in its paring pixel selection strategy: only pixels immediately adjacent to the central one are eligible, therefore imposing stronger spatial regularization. Additionally, with off-the-shelf classification sub-network designs, the proposed multi-stream, late-fusion CNN-based framework outperforms competing ones without requiring extensive network configuration tuning. Experimental results on three publicly available datasets demonstrate the performance of the proposed SPPF-based HSI classification framework.

To build small and efficient neural networks for hyperspectral image (HSI) classification, this letter presents a depthwise separable residual neural network (ResNet). This approach, motivated by the popular MobileNet architecture, decomposes the traditional spatial-spectral convolution operation into a spatial-independent pointwise spectral convolution and a spectral-independent depthwise spatial convolution. It allows the separation of spectral and spatial information in HSI and also greatly reduces the network size to prevent the overfitting issue. To better preserve the class boundaries and edges, the proposed ResNet is integrated into a maximum a posteriori (MAP) framework to allow the use of the conditional random field (CRF) model. The experiment results on benchmark HSI scene demonstrate that the proposed ResNet compares favorably with several popular deep learning HSI classifiers and that the ResNet-CRF approach achieves higher accuracy and better boundaries among neighboring classes.

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial–spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial–spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial–spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3 approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training.

Semisupervised learning (SSL) is often used when the number of labeled samples is very small compared to the number of unlabeled samples. It permits the exploitation of structure within unlabeled samples during the learning task. Like many other applications, remote sensing images suffer from the limited number of ground-truth samples and therefore semisupervised techniques may be used to overcome this limitation. In this paper, a semisupervised framework is proposed for classification of hyperspectral images with scarce labeled samples. Our method, which we call SSL-CC, utilizes fuzzy spectral clustering to label spatially neighboring samples. SSL-CC is implemented and tested on two benchmark hyperspectral datasets. Fuzzy clustering is compared to traditional crisp clustering (k-means) and the obtained results indicate that fuzzy clustering can significantly improve classification accuracy. SSL-CC achieves on average 60% improvement over a baseline SVM classifier.

In this paper, we propose a novel semi-supervised learning classification framework using box-based smooth ordering and multiple 1D-embedding-based interpolation (M1DEI) in [J. Wang, Semi-supervised learning using multiple one-dimensional embedding-based adaptive interpolation, Int. J. Wavelets Multiresolut. Inf. Process. 14(2) (2016) 11 pp.] for hyperspectral images. Due to the lack of labeled samples, conventional supervised approaches cannot generally perform efficient enough. On the other hand, obtaining labeled samples for hyperspectral image classification is difficult, expensive and time-consuming, while unlabeled samples are easily available. The proposed method can effectively overcome the lack of labeled samples by introducing new labeled samples from unlabeled samples in a label boosting framework. Furthermore, the proposed method uses spatial information from the pixels in the neighborhood of the current pixel to better catch the features of hyperspectral image. The proposed idea is that, first, we extract the box (cube data) of each pixel from its neighborhood, then apply multiple 1D interpolation to construct the classifier. Experimental results on three hyperspectral data sets demonstrate that the proposed method is efficient, and outperforms recent popular semi-supervised methods in terms of accuracies.

Hyperspectral image (HSI) classification methods based on deep learning techniques have succeeded wildly. However, the high-dimensional non-linearity, spectral mixing, and difficulty in labeling training samples of HSI still hinder the accuracy of HSI classification. Several patch-free-based methods were proposed for HSI classification and have attracted attention. Nevertheless, recent patch-free methods have focused on low- or high-order interactions, ignoring the abundant middle-order interactions, failing to capture the multi-order interactions in the context of HSI, and difficulty extracting discriminative features when the sample data is imbalanced. In this paper, a combining multi-attention and asymmetric convolution (CMAAC) global learning framework was proposed for insufficient utilization of spectral-spatial information and imbalanced HSI samples. In CMAAC, the channel convolutional long short-term mechanism and multi-order spatial aggregation block (CLMS), which aim to capture multi-order interactions information and extract more discriminant spectral-spatial features effectively, is proposed. The pyramid-enhanced attention mechanism (PEAM) alleviates the information loss in feature flow, retains more spatial information, and better solves the challenge of scale diversity in different land-cover types. The asymmetric convolutional structure (ACS) at the end of the model highlights the influence of local key feature points and improves its representation ability. Additionally, the framework introduces a joint loss function to reduce misclassified classes caused by the imbalance between well-classified and hard-classified samples. We conducted experiments on four benchmark datasets and compared them with the state-of-the-art approaches, demonstrating the model’s effectiveness and superiority.

A Binary Grey Wolf Optimization based Hybrid Convolutional Neural Network (BGWOHCNN) framework for hyperspectral image classification

A hyperspectral image classification framework and its application

Combining spectral and spatial information can significantly improve the classification performance of hyperspectral image (HSI). Currently, a lot of spectral–spatial HSI classification methods have been proposed. However, the task of HSI classification has remained challenging since the number of training samples is limited in real scenarios. In this article, we propose a novel HSI classification framework with single sample, in which the spectral self-similarity and spatial polygon structure information are fully combined to improve the classification performance. On the one hand, spectral self-similarity is used to expand training samples, which makes it possible to obtain sufficient samples with minimal cost. On the other hand, polygonal partition is introduced to acquire the geometrical structure of land covers in man-made environments. Specifically, the edge information of geometric objects is captured by polygonal partition, which can be utilized to constrain the spatial range of sample expansion and optimize the classification results. Experimental results on three real HSIs illustrate that the proposed method performs very well under small training sample size even when the number of samples is single per class.

Most of the existing spatial-spectral-based hyperspectral image classification (HSIC) methods mainly extract the spatial-spectral information by combining the pixels in a small neighborhood or aggregating the statistical and morphological characteristics. However, those strategies can only generate shallow appearance features with limited representative ability for classes with high interclass similarity and spatial diversity and therefore reduce the classification accuracy. To this end, we present a novel HSIC framework, named deep multiscale spatial-spectral feature extraction algorithm, which focuses on learning effective discriminant features for HSIC. First, the well pretrained deep fully convolutional network based on VGG-verydeep-16 is introduced to excavate the potential deep multiscale spatial structural information in the proposed hyperspectral imaging framework. Then, the spectral feature and the deep multiscale spatial feature are fused by adopting the weighted fusion method. Finally, the fusion feature is put into a generic classifier to obtain the pixelwise classification. Compared with the existing spectral-spatial-based classification techniques, the proposed method provides the state-of-the-art performance and is much more effective, especially for images with high nonlinear distribution and spatial diversity.

A spatial-spectral metric learning (SSML) framework for hyperspectral image (HSI) classification is proposed. SSML learns a metric by considering both the spectral characteristics and spatial features represented as the mean of neighboring pixels. It first performs the local pixel neighborhood preserving embedding (LPNPE) to reduce the dimensionality of HSI and meanwhile to preserve the spatial local similarity structure. Then, it learns a spectral and spatial distance metric, separately. Finally, the combination of the spectral and spatial metrics yields a joint spatial-spectral metric. It is followed by a nearest neighbor (NN) classifier for HSI classification. SSML shows good performance over the spectral and spatial NN and SVM on the benchmark hyperspectral data set of Indian Pines.

Hyperspectral image classification has been widely applied in many fields, but it also faces challenges because of small number of labeled samples. In this paper, we propose the Multiscale Multistage Spectral-Spatial Feature Fusion Framework (M^2 S^2 F^2 ) for hyperspectral image classification using small training samples. The Framework is the combination of two deep convolutional neural networks, which can extract more representative and discriminative features by combining the following operations. Firstly, two different scale 3-D cubes are the inputs for the spectral and spatial feature extraction respectively. Secondly, by fusing strong complementary information between different layers, we form multistage spectral and spatial features by fusion primary, intermediate and advanced features of the spectral and spatial features respectively. Spectral and spatial features are extracted by spectral and spatial skipped residual blocks, which can effectively alleviate the problems of gradient degradation. Thirdly, the fusion of complementary multistage spectral and spatial features can improve the classification accuracy. Experimental results on the IN, UP and KSC datasets show the effectiveness of the proposed method using small training samples.

Hyperspectral image (HSI) classification has received extensive attention by the development of deep learning and has achieved great success. However, most of the deep learning-based approaches tend to extract features of spatial content by disrupting spectral information or to extract sequential spectral features in short-range context. On the other hand, Transformers-based models address this problem by learning long-range relationship. This study introduces a novel spectral-swin transformer (SpectralSWIN) network. The proposed network effectively projects the HSI data from spectral characteristics into spatial and spectral feature representation. Specifically, SpectralSWIN network makes use of a newly proposed swin-spectral module (SSM) for processing the spatial and spectral features concurrently. As far as we know, this is the first time that a transformer backbone designed for vision domain has been proposed for HSI classification. Extensive experiments conducted on two different HSI prove the superiority and effectiveness of the proposed method over the state-of-the-art methods in terms of both quantitative and visual evaluations.

The difficulty existing in synthetic aperture radar (SAR) image classification is large amounts of unpredictable and inestimable speckle, leading to degradation of the image quality and concealing important objectives of interest. By exploiting an efficient image features extraction technique, bag-of-visual-words (BOV) for its ability of 'midlevel' feature representation, and a new developed non-local (NL-) means denosing method suitable for multiplicative speckle, we present a novel and effective BOV framework for SAR image classification. Compared with the other two representative algorithms, the experimental results show that the proposed algorithm has obtained more satisfactory and cogent classification performance and performed more robustness to SAR speckle.

Mamba-based spatial-spectral fusion network for hyperspectral unmixing