Hyperspectral Image Classification Using Spectral-Spatial LSTMs

Recurrent neural networks (RNNs) have been widely used for hyperspectral image (HSI) classification via sequence modeling. However, most of the RNN methods focus on modeling long-range dependencies along the spectral direction, without fully exploring multi-directional dependencies in the joint spectral-spatial domain. To tackle this issue, we propose MSLAN, a two-branch multi-directional spectral-spatial long short-term memory (LSTM) attention network, for HSI classification. In particular, we employ LSTMs to extract six-directional spatial-spectral features which simultaneously capture the spectral-spatial dependencies along different directions. We then design an attention-based feature fuse module to integrate these directional features, followed by a fully connected layer with cross-entropy loss for classification. Additionally, we incorporate an auxiliary branch into our model to enhance the generalization capability. In this branch, random spatial shuffle and a cosine loss are explored for feature consistency learning by taking into account the varying spatial distributions. The resulting two branch networks, sharing the same network structure and weights, are incorporated into a unified deep learning architecture for training. Experiments show the superiority of MSLAN to the state-of-the-art methods for HSI classification with limited training samples.

Hyperspectral Image (HSI) classification plays a key role remote sensing field. Recently, deep learning has demonstrated its effectiveness in HSI Classification field. This paper presents a spectral-spatial HSI classification technique established on the deep learning based deep belief network (DBN) for deep and abstract feature extraction and adaptive boundary adjustment based segmentation. Proposed approach focuses on integrating the deep learning based spectral features and segmentation based spatial features into a framework for improved performance. Specifically, first the deep DBN model is exploited as a spectral feature extraction based classifier to extract the deep spectral features. Second, spatial contextual features are obtained by utilizing effective adaptive boundary adjustment based segmentation technique. Finally, maximum voting based criteria is operated to integrate the results of extracted spectral and spatial information for improved HSI classification. In general, exploiting spectral features from DBN process and spatial features from segmentation and integration of spectral and spatial information by maximum voting based criteria, has a substantial effect on the performance of HSI classification. Experimental performance on real and widely used hyperspectral data sets with different contexts and resolutions demonstrates the accuracy of the proposed technique and performance is comparable to several recently proposed HSI classification techniques.

Hyperspectral image (HSI) classification has become one of the most significant tasks in the field of hyperspectral analysis. However, classifying each pixel in HSI accurately is challenging due to the curse of dimensionality and limited training samples. In this paper, we present an HSI classification architecture called camera spectral response network (CSR-Net), which can learn the optimal camera spectral response (CSR) function for HSI classification problems and effectively reduce the spectral dimensions of HSI. Specifically, we design a convolutional layer to simulate the capturing process of cameras, which learns the optimal CSR function for HSI classification. Then, spectral and spatial features are further extracted by spectral and spatial attention modules. On one hand, the learned CSR can be implemented physically and directly used to capture scenes, which makes the image acquisition process more convenient. On the other hand, compared with ordinary HSIs, we only need images with far fewer bands, without sacrificing the classification precision and avoiding the curse of dimensionality. The experimental results of four popular public hyperspectral datasets show that our method, with only a few image bands, outperforms state-of-the-art HSI classification methods which utilize the full spectral bands of images.

For hyperspectral image (HSI) classification, two branch networks generally use the convolution neural networks (CNNs) to extract the spatial features and the long short-term memory (LSTM) to learn the spectral features. However, CNN with a local kernel neglects the global properties of the whole HSI. LSTM doesn't consider the macroscopic and detailed information of spectra. In this paper, we propose a dual-view spectral and global spatial feature fusion network (DSGSF) to extract the spatial-spectral features for HSI classification, including a spatial subnetwork and a spectral subnetwork. In the spatial subnetwork, we propose a global spatial feature representation model based on the encoder-decoder structure with channel attention and spatial attention to learn the global spatial features. In the spectral subnetwork, we design a dual-view spectral feature aggregation model with view attention to learn the diversity of spectral features. By fusing the two subnetworks, we construct DSGSF to extract the spatial-spectral features of HSI with strong discriminating performance. Experimental results on three public datasets illustrate that the proposed method can achieve competitive results compared with the state-of-the-art methods. Code: https://github.com/RZWang-WH/DSGSF.

Most hyperspectral image (HSI) classification methods assume that all classes in the test set are present during training. However, in real-world applications, acquiring labeled training samples is challenging. As a result, it is difficult for the training dataset to cover all possible land cover types, leading to the generalized zero-shot learning (GZSL) problem. Recently, vision-language models (VLMs) have provided rich semantic priors for land cover classes, offering promising potential for GZSL. However, two fundamental gaps hinder their application to HSI classification: the task paradigm gap, arising from the difference between image-level VLMs and the pixel-level HSI classification task; and the knowledge gap, due to the inconsistency between VLM features and HSI spectral–spatial representations. To bridge both gaps, a novel framework leveraging VLM semantic priors for GZSL in HSI classification is proposed, primarily using pseudo-labeling technique to provide knowledge for unseen classes. Specifically, a pseudo-label generation and enhancement module enables a paradigm transition from image-level understanding to pixel-level classification by incorporating HSI’s spatial information. A pseudo-label correction module then refines noisy labels using spectral cues to address the knowledge gap. Finally, a global learning strategy integrates pseudo-label distillation, supervised learning, and feature regularization to classify seen classes while enabling generalization to unseen ones. Experiments on benchmark HSI datasets demonstrate the proposed method’s superiority in generalized zero-shot classification. This work highlights the potential of VLMs in advancing HSI classification in practical applications.

Classification of hyperspectral images always suffers from high dimensionality and very limited labeled samples. Recently, the spectral-spatial classification has attracted considerable attention and can achieve higher classification accuracy and smoother classification maps. In this paper, a novel spectral-spatial classification method for hyperspectral images by using kernel methods is investigated. For a given hyperspectral image, the principle component analysis (PCA) transform is first performed. Then, the first principle component of the input image is segmented into non-overlapping homogeneous regions by using the entropy rate superpixel (ERS) algorithm. Next, the local spectral histogram model is applied to each homogeneous region to obtain the corresponding texture features. Because this step is performed within each homogenous region, instead of within a fixed-size image window, the obtained local texture features in the image are more accurate, which can effectively benefit the improvement of classification accuracy. In the following step, a contextual spectral-texture kernel is constructed by combining spectral information in the image and the extracted texture information using the linearity property of the kernel methods. Finally, the classification map is achieved by the support vector machines (SVM) classifier using the proposed spectral-texture kernel. Experiments on two benchmark airborne hyperspectral datasets demonstrate that our method can effectively improve classification accuracies, even though only a very limited training sample is available. Specifically, our method can achieve from 8.26% to 15.1% higher in terms of overall accuracy than the traditional SVM classifier. The performance of our method was further compared to several state-of-the-art classification methods of hyperspectral images using objective quantitative measures and a visual qualitative evaluation.

Spectral features cannot effectively reflect the differences among the ground objects and distinguish their boundaries in hyperspectral image (HSI) classification. Multi-scale feature extraction can solve this problem and improve the accuracy of HSI classification. The Gaussian pyramid can effectively decompose HSI into multi-scale structures, and efficiently extract features of different scales by stepwise filtering and downsampling. Therefore, this paper proposed a Gaussian pyramid based multi-scale feature extraction (MSFE) classification method for HSI. First, the HSI is decomposed into several Gaussian pyramids to extract multi-scale features. Second, we construct probability maps in each layer of the Gaussian pyramid and employ edge-preserving filtering (EPF) algorithms to further optimize the details. Finally, the final classification map is acquired by a majority voting method. Compared with other spectral-spatial classification methods, the proposed method can not only extract the characteristics of different scales, but also can better preserve detailed structures and the edge regions of the image. Experiments performed on three real hyperspectral datasets show that the proposed method can achieve competitive classification accuracy.

In recent studies, superpixel segmentation has been integrated into hyperspectral (HS) image classification methods. However, the existing superpixel-based classification methods usually suffer from two serious problems. First, the accuracy and efficiency of current superpixel segmentation approaches cannot meet the demands of practical applications for HS images; second, conventional superpixel-based classification methods generally consider each generated superpixel as a unit for the image classification, which may help to reduce the computing time but result in a significant decrease of the classification accuracy. To solve the problems, we propose a fast region growing based superpixel segmentation (FRGSS) algorithm and a novel texture-adaptive superpixel integration strategy (TASIS) for the HS image classification. Experimental results on real Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) HS images demonstrate that the proposed FRGSS outperforms the state-of-the-art superpixel algorithm. In addition, the superiority of the TASIS is verified compared to the pixel-wise and the conventional superpixel-based classification methods.

Hyperspectral image (HSI) classification has attracted much attention and extensive research efforts over the past decade. Due to few labeled samples versus high dimensional features, it is a challenging problem in practice. Recently, combining the pixel spectral information and the spatial (neighborhood) information has been verified to be effective for HSI classification. In this paper, we introduce a novel method for HSI classification using set-to-set distance (SSD). Based on the assumption that neighbor pixels tend to belong to the same class with high probability, we model a test pixel and its neighbor pixels as a testing set (or a neighbor set) inspired by bilateral filtering. Meanwhile, the training pixels belong to the same class are modeled as a training set. Therefore, the classification is based on comparisons of sets distances. Experiments on a real HSI dataset show that our proposed method outperforms a number of existing state-of-the-art approaches.

Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples

Recently, hyperspectral image (HSI) classification methods based on deep learning have demonstrated excellent performance. However, these deep learning methods still face two major challenges. One is that they require a large number of labeled samples, and the other is that training parameters takes a lot of time. In this paper, we propose a dual-branch deep stochastic adaptive Fourier decomposition (SAFD) network (DSAFDNet) to alleviate the aforementioned two issues in HSI classification applications. SAFD is a newly developed signal processing tool with solid mathematical foundation. It can be used to find common filters (i.e. convolution kernels) of a set of random signals or multi-signals. Since the convolution kernels obtained by SAFD decomposition are complex numbers, few deep learning methods directly deal with such complex convolution kernels. To this end, we propose a dual-branch network to extract deep features from hyperspectral images using both real and imaginary parts of convolutional kernels. After deep feature extraction using DSAFDNet, we further investigate the classification performance of different classifiers on the extracted features. Experimental results show that the proposed method outperforms some HSI classification methods with similar principles. Moreover, compared with other state-of-the-art deep learning methods, the proposed method can achieve better classification performance.

Recently, collaborative representation has received much attention in the hyperspectral image (HSI) classification due to its simplicity and effectiveness. However, the existing collaborative representation-based HSI classification methods ignore the correlation among different classes. To overcome this problem, we propose a discriminative kernel collaborative representation and Tikhonov regularization method (DKCRT) for HSI classification, which can make the kernel collaborative representation of different classes to be more discriminative. Specifically, the kernel trick is adopted to map the original HSI into a high space to improve the class separability. Besides, distance-weighted kernel Tikhonov regularization is adopted to enforce these training samples to have large representation coefficients, which are similar to the test sample in the high-dimensional feature space. Moreover, we add a discriminative regularization term to further enhance the separability of different classes, which can take the correlation among different classes into consideration. Furthermore, to take the spatial information of HSI into consideration, we extend the DKCRT to a joint version named JDKCRT. Experiments on real HSIs demonstrate the efficiency of the proposed DKCRT and JDKCRT.

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

The hyperspectral image (HSI) classification applications become widespread in multiple applications. Therefore, the accurate classification of ground features using HSIs is an important research topic that has received a lot of interest. However, the conventional approaches failed to classify the pixels perfectly, which resulted in reduced accuracy. Thus, this work is focused on implementation of HSI classification using generative adversarial networks (HCGAN-Net). Initially, Gabor filtering is applied on HSIs for elimination of different types of noises, which also extracts the spatial features. Then, probabilistic principal component analysis (PCA) is applied to perform the dimensionality reduction operation, which also selects the best features using probability dependencies between spatial features. Then, spatial features are extracted for each spectral band and forms the hybrid spatial-spectral features. Finally, HCGAN-Net is used to performs the HSI classification operation using trained spatial-spectral features. The simulation results show that the proposed HCGAN-Net resulted in superior subjective and objective performance as compared to state of art HSI classifiers on various datasets.

Deep learning methods, especially convolutional neural networks(CNN), have been widely used in hyperspectral image(HSI) classification. Recently, graph convolutional networks (GCN) have shown great potential in HSI classification problem. However, the existing GCN-based methods have several problems. First, the existing methods rely too much on the adjacency matrix, which cannot be changed during training. Furthermore, most of them can only use a single kind of feature, and fail to extract the spectral-spatial information from the HSI. Finally, for the existing GCN-based methods, it is difficult to achieve the same accuracy as the mature CNN methods. In this paper, we propose a spectral-spatial hypergraph convolutional neural network (S²HCN) for HSI classification. Compared with the existing GCN-based methods, S²HCN has the following advantages. Different from the adjacency matrix that is fixed during training of GCN, S²HCN can dynamically update the weight of the hyperedge during training, which reduces the reliance on prior information to a certain extent. In addition, S²HCN generates hyperedges from the spectral and spatial features independently, and adopts the incidence matrix composed of all hyperedges to replace the adjacency matrix in GCN. In this way, the spectral and spatial features can be better integrated. Finally, compared to a simple graph structure, the hypergraph structure can express the high-dimensional relationships in the data, which is beneficial to classification problems. Sufficient experiments on two popular HSI datasets have proved the effectiveness of S²HCN.