GATSiamNet: a dual branch network for few-shot hyperspectral image classification

In recent years, the integration of convolutional neural networks (CNNs) and Transformers has significantly advanced hyperspectral image (HSI) classification by jointly capturing local and global features. However, most existing methods primarily focus on the fusion of spectral–spatial features while neglecting the complementary information contained in frequency domain features. To address this issue, we propose a spatial frequency cross-attention fusion network (SFCFNet) that jointly models spectral, spatial, and frequency-domain features for HSI classification. The framework consists of three core modules: first, the multiscale spectral spatial feature learning (MSSFL) module extracts joint spectral spatial features using multi-scale 3D and 2D convolutions. Next, the triple-branch representation module (TBRM) employs three branches to capture global spatial features of large-scale structures, local spatial features of fine grained textures, and multi-scale frequency features based on Haar wavelet decomposition, providing complementary multi domain representations for subsequent deep fusion. Finally, the dual-domain feature cross-attention fusion module (DFCF) achieves effective fusion of spatial structures and frequency domain textures, enhancing the model's ability to separate complex backgrounds from fine-grained targets and thereby improving classification performance. Compared with other methods, SFCFNet achieves higher overall accuracy (OA) on the Salinas, Houston2013, WHU-Hi-LongKou, and Xuzhou datasets, reaching 99.05%, 98.07%, 98.76%, and 98.18%, respectively.

Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

The lack of training samples remains one of the major obstacles in applying convolutional neural networks (CNNs) to the hyperspectral image (HSI) classification. In this letter, the accurate classification of HSI with limited training samples is investigated. Due to the advantages of minimizing the distance between samples in the same class and maximizing the distance between samples in different classes, siamese CNN is used for HSI classification with limited training samples. After that, to improve the classification performance, data augmentation is investigated for siamese CNN-based HSI classification. Specifically, pair data augmentation based on CutMix is proposed to generate the training pairs of the same or different classes and the new generated training pairs are used to train siamese CNN. Traditional data augmentation methods simply generate new training samples. It is not proper when data augmentation methods keep the loss function and change the content of the input at the same time. Therefore, a soft-loss-based siamese CNN, which changes its loss according to the coupled replacement data augmentation, is proposed to further address the HSI classification with limited training samples. In the experimental part, on two widely used hyperspectral datasets, the influences of different training samples and window sizes are discussed, and the experimental results reveal that the proposed soft-augmentation-based siamese CNN provides competitive results with limited training samples compared with state-of-the-art methods.

Traditional hyperspectral image (HSI) classification methods mainly include machine learning and convolutional neural network. However, they extremely depend on the large training samples. To obtain high accuracy on limited training samples, we propose a novel end-to-end spectral-spatial multi-scale network (SSMSN) for HSI classification. The SSMSN uses the multi-scale spectral module and the multi-scale spatial module to extract discriminative multi-scale spectral and multi-scale spatial features separately. In the multi-scale spectral module and spatial module, the multi-scale Res2Net block structure can learn multi-scale features at a granular level and increase the range of receptive fields by constructing the hierarchical residual-like connection within one single residual block. To alleviate the overfitting problem and further improve the classification accuracy on limited training samples, we adopt a simple but effective hinge cross-entropy loss function to train the SSMSN at the dynamic learning rate. A large number of experimental results demonstrate that on the Indiana Pines, University of Pavia, Kennedy Space Center, and Salinas Scene data sets, the proposed SSMSN achieves higher classification accuracy than state-of-the-art methods on limited training samples. Meanwhile, our SSMSN obtains less training and testing time than the popular AUSSC method.

For hyperspectral image (HSI) classification, two branch networks generally use the convolution neural networks (CNNs) to extract the spatial features and the long short-term memory (LSTM) to learn the spectral features. However, CNN with a local kernel neglects the global properties of the whole HSI. LSTM doesn't consider the macroscopic and detailed information of spectra. In this paper, we propose a dual-view spectral and global spatial feature fusion network (DSGSF) to extract the spatial-spectral features for HSI classification, including a spatial subnetwork and a spectral subnetwork. In the spatial subnetwork, we propose a global spatial feature representation model based on the encoder-decoder structure with channel attention and spatial attention to learn the global spatial features. In the spectral subnetwork, we design a dual-view spectral feature aggregation model with view attention to learn the diversity of spectral features. By fusing the two subnetworks, we construct DSGSF to extract the spatial-spectral features of HSI with strong discriminating performance. Experimental results on three public datasets illustrate that the proposed method can achieve competitive results compared with the state-of-the-art methods. Code: https://github.com/RZWang-WH/DSGSF.

Deep learning techniques have boosted the performance of hyperspectral image (HSI) classification. In particular, convolutional neural networks (CNNs) have shown superior performance to that of the conventional machine learning algorithms. Recently, a novel type of neural networks called capsule networks (CapsNets) was presented to improve the most advanced CNNs. In this paper, we present a modified two-layer CapsNet with limited training samples for HSI classification, which is inspired by the comparability and simplicity of the shallower deep learning models. The presented CapsNet is trained using two real HSI datasets, i.e., the PaviaU (PU) and SalinasA datasets, representing complex and simple datasets, respectively, and which are used to investigate the robustness or representation of every model or classifier. In addition, a comparable paradigm of network architecture design has been proposed for the comparison of CNN and CapsNet. Experiments demonstrate that CapsNet shows better accuracy and convergence behavior for the complex data than the state-of-the-art CNN. For CapsNet using the PU dataset, the Kappa coefficient, overall accuracy, and average accuracy are 0.9456, 95.90%, and 96.27%, respectively, compared to the corresponding values yielded by CNN of 0.9345, 95.11%, and 95.63%. Moreover, we observed that CapsNet has much higher confidence for the predicted probabilities. Subsequently, this finding was analyzed and discussed with probability maps and uncertainty analysis. In terms of the existing literature, CapsNet provides promising results and explicit merits in comparison with CNN and two baseline classifiers, i.e., random forests (RFs) and support vector machines (SVMs).

Hyperspectral images (HSIs) classification methods based on convolutional neural networks are becoming increasingly popular. Many proposed methods extract spatial and spectral features simultaneously, and the interaction between the two types of features leads to unsatisfied classification results. Moreover, most of the existing CNN-based methods mainly consider single-scale, which can cause some important information to be neglected. To address the aforementioned two issues, we propose a Attention-Based Multi-Scale Network (AMSN) for HSIs classification. First, the proposed network is based on 3D-CNN, through channel branch and spatial branch, the AMSN can capture more distinctive spatial and spectral features using different convolution kernels, respectively. Second, Local and global features are extracted by dense network. Then, two features are concatenated to make full use of multi-scale features. Third, attention blocks are used after each branch to achieve the most distinctive features. Experimental results on three HSIs datasets demonstrate that the proposed framework can achieve better classification performance than the several state-of-the-art methods.

In recent years, deep-learning-based hyperspectral image (HSI) classification networks have become one of the most dominant implementations in HSI classification tasks. Among these networks, convolutional neural networks (CNNs) and attention-based networks have prevailed over other HSI classification networks. While convolutional neural networks with perceptual fields can effectively extract local features in the spatial dimension of HSI, they are poor at capturing the global and sequential features of spectral–spatial information; networks based on attention mechanisms, for example, Transformer, usually have better ability to capture global features, but are relatively weak in discriminating local features. This paper proposes a fusion network of convolution and Transformer for HSI classification, known as FusionNet, in which convolution and Transformer are fused in both serial and parallel mechanisms to achieve the full utilization of HSI features. Experimental results demonstrate that the proposed network has superior classification results compared to previous similar networks, and performs relatively well even on a small amount of training data.

Convolutional neural networks (CNNs), a kind of feedforward neural network with a deep structure, are one of the representative methods in hyperspectral image (HSI) classification. However, redundant information and interclass interference are common and challenging problems in HSI classification. In addition, if the spectral and spatial information is not properly extracted and analyzed, it will affect the classification performance of the network to a great extent. Aiming at these issues, this article proposes an HSI classification method based on an adaptive hash attention mechanism and a lower triangular network (AHA-LT). First, the attention mechanism is introduced in the preprocessing stage, which is composed of the spectral attention module and the adaptive hash spatial attention module in series. Then, the data processed by the attention mechanism are introduced into the lower triangular network (LTNet) to obtain the fused high-dimensional semantic features. Finally, we compress the features and obtain the output classification results through several fully connected layers. Among them, LTNet is composed of 2-D–3-D CNN and multiscale features. The network integrates the characteristics of multibranch, feature fusion, feature compression, and skip connections. Extensive experiments on four widely used HSI data sets show that the proposed method can obtain a great improvement in performance compared with the existing methods.

In the research of hyperspectral image classification, deep learning-based methods have achieved promising results. However, due to the high cost and difficulty in obtaining labeled hyperspectral image samples, traditional deep learning methods experience a significant performance decline under few-shot conditions. This paper proposes a deep few-shot learning method, referred to as Wavelet-Based Multi-stream Spatial-Frequency Fusion Network, to address the issue of small sample size in hyperspectral image classification. Through a multi-stream structure, features at various scales are obtained, effectively alleviating the issue of insufficient feature representation caused by limited training samples. These features are then fed into a multi-level wavelet transformation module integrated with a frequency-domain attention module. By decomposing the input into high-frequency and low-frequency components via wavelet transformation, and subsequently inputting these components into the frequency-domain attention module, a richer representation of features in the frequency domain is achieved. Finally, a Spatial-Frequency Fusion module is employed to not only extract abundant feature information but also fuse features from different scales. This process results in feature maps that encompass more comprehensive global feature information, thereby significantly enhancing classification accuracy under small sample conditions. The proposed method has been compared with traditional machine learning approaches and recent deep learning methods on the Houston, Dioni, and University of Pavia datasets. Under training conditions using only 1 to 5, 10, 15, and 20 samples per class, the experimental results demonstrate the superiority of the method proposed in this paper.

Due to a lack of labeled samples, deep learning methods generally tend to have poor classification performance in practical applications. Few-shot learning (FSL), as an emerging learning paradigm, has been widely utilized in hyperspectral image (HSI) classification with limited labeled samples. However, the existing FSL methods generally ignore the domain shift problem in cross-domain scenes and rarely explore the associations between samples in the source and target domain. To tackle the above issues, a graph-based domain adaptation FSL (GDAFSL) method is proposed for HSI classification with limited training samples, which utilizes the graph method to guide the domain adaptation learning process in a uniformed framework. First, a novel deep residual hybrid attention network (DRHAN) is designed to extract discriminative embedded features efficiently for few-shot HSI classification. Then, a graph-based domain adaptation network (GDAN), which combines graph construction with domain adversarial strategy, is proposed to fully explore the domain correlation between source and target embedded features. By utilizing the fully explored domain correlations to guide the domain adaptation process, a domain invariant feature metric space is learned for few-shot HSI classification. Comprehensive experimental results conducted on three public HSI datasets demonstrate that GDAFSL is superior to the state-of-the-art with a small sample size.

Hyperspectral image (HSI) classification has become one of the most significant tasks in the field of hyperspectral analysis. However, classifying each pixel in HSI accurately is challenging due to the curse of dimensionality and limited training samples. In this paper, we present an HSI classification architecture called camera spectral response network (CSR-Net), which can learn the optimal camera spectral response (CSR) function for HSI classification problems and effectively reduce the spectral dimensions of HSI. Specifically, we design a convolutional layer to simulate the capturing process of cameras, which learns the optimal CSR function for HSI classification. Then, spectral and spatial features are further extracted by spectral and spatial attention modules. On one hand, the learned CSR can be implemented physically and directly used to capture scenes, which makes the image acquisition process more convenient. On the other hand, compared with ordinary HSIs, we only need images with far fewer bands, without sacrificing the classification precision and avoiding the curse of dimensionality. The experimental results of four popular public hyperspectral datasets show that our method, with only a few image bands, outperforms state-of-the-art HSI classification methods which utilize the full spectral bands of images.

In recent years, convolutional neural networks (CNNs) have been widely used in hyperspectral image classification and have achieved good performance. However, the high dimensions and few samples of hyperspectral remote sensing images tend to be the main factors restricting improvements in classification performance. At present, most advanced classification methods are based on the joint extraction of spatial and spectral features. In this article, an improved dense block based on a multiscale spectral pyramid (MSSP) is proposed. This method uses the idea of multiscale and group convolution of the convolution kernel, which can fully extract spectral information from hyperspectral images. The designed MSSP is the main unit of the spectral dense block (called MSSP Block). Additionally, a short connection with nonlinear transformation is introduced to enhance the representation ability of the model. To demonstrate the effectiveness of the proposed dual-branch multiscale spectral attention network, some experiments are conducted on five commonly used datasets. The experimental results show that, compared with some state-of-the-art methods, the proposed method can provide better classification performance and has strong generalization ability.

Recently, deep learning-based methods have made great progress in hyperspectral image (HSI) classification (HSIC). Different from ordinary images, the intrinsic complexity of HSIs data still limits the performance of many common convolutional neural network (CNN) models. Thus, the network architecture becomes more and more complex to extract discriminative spectral-spatial features. For instance, 3-D CNN usually has a large number of trainable parameters, thus increasing the computational complexity of the HSIC. In this letter, we designed a cooperative spectral-spatial attention dense network (CS <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ADN) that takes raw 3-D HSI data as input data. Specifically, the attention module consists of spectral and spatial axes, by which the salient spectral-spatial features will be emphasized. Furthermore, we combined these attention modules with the dense connection, which is termed as the lightweight dense block; it has a lower computation cost and achieves better classification performance. At the same time, we introduced the center loss, by jointly using the supervision of the center loss and the softmax loss, where the discriminative features could be clearly observed, particularly for small data sets. Experimental results on the biased and unbiased HSI data show that our method outperforms several state-of-the-art methods in HSIC with small training samples.