Attention-based multiscale deep learning with unsampled pixel utilization for hyperspectral image classification

Hyperspectral sensors provide an opportunity to capture the intensity of high spatial/spectral information and enable applications for high-level earth observation missions, such as accurate land cover mapping and target/object detection. Currently, convolutional neural networks (CNNs) are good at coping with hyperspectral image processing tasks because of the strong spatial and spectral feature extraction ability brought by hierarchical structures, but the convolution operation in CNNs is limited to local feature extraction in both dimensions. In the meanwhile, the introduction of the Transformer structure has provided an opportunity to capture long-distance dependencies between tokens from a global perspective; however, Transformer-based methods have a restricted ability to extract local information because they have no inductive bias, as CNNs do. To make full use of these two methods’ advantages in hyperspectral image processing, a dual-flow architecture named Hyper-LGNet to couple local and global features is firstly proposed by integrating CNN and Transformer branches to deal with HSI spatial-spectral information. In particular, a spatial-spectral feature fusion module (SSFFM) is designed to maximally integrate spectral and spatial information. Three mainstream hyperspectral datasets (Indian Pines, Pavia University and Houston 2013) are utilized to evaluate the proposed method’s performance. Comparative results show that the proposed Hyper-LGNet achieves state-of-the-art performance in comparison with the other nine approaches concerning overall accuracy (OA), average accuracy (AA) and kappa index. Consequently, it is anticipated that, by coupling CNN and Transformer structures, this study can provide novel insights into hyperspectral image analysis.

Hyperspectral image (HSI) classification has many applications in different diverse research fields. We propose a method for HSI classification using principal component analysis (PCA), 2D spatial convolution, and support vector machine (SVM). Our method takes advantage of correlation in both spatial and spectral domains in an HSI data cube at the same time. We use PCA to reduce the dimensionality of an HSI data cube. We then perform spatial convolution to the dimension-reduced data cube once and then to the convolved data cube for the second time. As a result, we have generated two convolved PCA output data cubes in a multiresolution way. We feed the two convolved data cubes to SVM to classify each pixel to one of the known classes. Experiments on three widely used hyperspectral data cubes (i.e., Indian Pines, Pavia University, and Salinas) demonstrate that our method can improve the classification accuracy significantly when compared to a few existing methods. Our method is relatively fast in terms of central processing unit computational time as well.

In recent years, the integration of convolutional neural networks (CNNs) and Transformers has significantly advanced hyperspectral image (HSI) classification by jointly capturing local and global features. However, most existing methods primarily focus on the fusion of spectral–spatial features while neglecting the complementary information contained in frequency domain features. To address this issue, we propose a spatial frequency cross-attention fusion network (SFCFNet) that jointly models spectral, spatial, and frequency-domain features for HSI classification. The framework consists of three core modules: first, the multiscale spectral spatial feature learning (MSSFL) module extracts joint spectral spatial features using multi-scale 3D and 2D convolutions. Next, the triple-branch representation module (TBRM) employs three branches to capture global spatial features of large-scale structures, local spatial features of fine grained textures, and multi-scale frequency features based on Haar wavelet decomposition, providing complementary multi domain representations for subsequent deep fusion. Finally, the dual-domain feature cross-attention fusion module (DFCF) achieves effective fusion of spatial structures and frequency domain textures, enhancing the model's ability to separate complex backgrounds from fine-grained targets and thereby improving classification performance. Compared with other methods, SFCFNet achieves higher overall accuracy (OA) on the Salinas, Houston2013, WHU-Hi-LongKou, and Xuzhou datasets, reaching 99.05%, 98.07%, 98.76%, and 98.18%, respectively.

A combination method of stacked autoencoder and 3D deep residual network for hyperspectral image classification

Deep learning has achieved remarkable success in hyperspectral image (HSI) classification, attributed to its powerful feature extraction capabilities. However, existing methods face several challenges: Convolutional Neural Networks (CNNs) are limited in modeling long-range spectral dependencies because of their limited receptive fields; Transformers are constrained by their quadratic computational complexity; and Mamba-based methods fail to fully exploit spatial–spectral interactions when handling high-dimensional HSI data. To address these limitations, we propose MRFP-Mamba, a novel Multi-Receptive-Field Parallel Mamba architecture that integrates hierarchical spatial feature extraction with efficient modeling of spatial–spectral dependencies. The proposed MRFP-Mamba introduces two key innovation modules: (1) A multi-receptive-field convolutional module employing parallel 1×1, 3×3, 5×5, and 7×7 kernels to capture fine-to-coarse spatial features, thereby improving discriminability for multi-scale objects; and (2) a parameter-optimized Vision Mamba branch that models global spatial–spectral relationships through structured state space mechanisms. Experimental results demonstrate that the proposed MRFP-Mamba consistently surpasses existing CNN-, Transformer-, and state space model (SSM)-based approaches across four widely used hyperspectral image (HSI) benchmark datasets: PaviaU, Indian Pines, Houston 2013, and WHU-Hi-LongKou. Compared with MambaHSI, our MRFP-Mamba achieves improvements in Overall Accuracy (OA) by 0.69%, 0.30%, 0.40%, and 0.97%, respectively, thereby validating its superior classification capability and robustness.

Hyperspectral image classification based on combining texture and multiscale guided filter feature of principal component

Due to the 3-D nature of hyperspectral images, as well as the spatial properties (such as regularity and continuity) of land covers, many 3-D feature extraction operators have been designed to fully exploit the joint spatial–spectral information. However, the large amount of obtained features can suffer from the “curse of dimensionality” problem, especially for the small training sample set. Moreover, various spatial–spectral features can represent the characteristics of the hyperspectral image from different aspects. In this paper, a multiple 3-D feature fusion framework (M3DF3) has been proposed for hyperspectral image classification. First, we extend the 2-D Gabor surface feature into 3-D (3DSF) domains to comply with the spatial–spectral structure of the hyperspectral image, which is directly applied on the original hyperspectral image instead of the Gabor features. Second, three 3-D feature extraction methods, including the 3-D morphological profile, the 3-D local binary pattern, and the proposed 3DSF, that, respectively, characterize the hyperspectral image from three different angles, i.e., morphology, local dependence, and shape smoothness, are fused under a multitask sparse representation framework to take full advantage of the multiple 3-D features together. The proposed M3DF3 approach was fully tested on three real-world hyperspectral image data, i.e., the widely used Indian Pines, Pavia University, and Houston University. The results show that our method can achieve as high as 68.22%, 79.44%, and 72.84% accuracies, respectively, even when only few samples, i.e., three samples per class, are used for training.

Hyperspectral image (HSI) classification is a vital but difficult task due to its significant spectral variability and nonlinear structure. Nowadays, complex spatial-spectral networks have achieved remarkable successes in HSI classification, but limited by the large complexity and hardware demands. Spectral networks with simple architectures alleviate this problem to some degree, however, most of them have downgraded performance as a result of insufficient excavation of spectral diagonal information and channel correlations. To overcome these problems, this paper proposes a fresh atrous convolution channel attention aided dense convolutional neural network (ACADCN) for HSI classification, which enhances the exploitation of spectral feature representations and channel correlations to provide a better classification with limited samples. On the one hand, an effective 1D dense block is constructed to deeply mine spectral discriminability by taking advantages of hierarchical representations and establish a deep 1D convolutional neural network, with the complementarity of different level features integrated. On the other hand, a singularly designed atrous convolution channel attention (ACA) module is used to learn multiscale cross-channel correlations to make up the locality of convolutions. The effectiveness of ACADCN is verified on two commonly used HSIs, with a mean overall accuracy (OA) of 94.09%, average accuracy (AA) of 94.63% and Kappa of 0.9254 achieved. The experimental results show its superiority to the other advanced deep spectral classifiers.

In the field of hyperspectral image classification, deep learning technology, especially convolutional neural networks, has achieved remarkable progress. However, convolutional neural network models encounter challenges in hyperspectral image classification due to limitations in their receptive fields. Conversely, the global modeling capability of Transformers has garnered attention in hyperspectral image classification. Nevertheless, the high computational cost and inadequate local feature extraction hinder its widespread application. In this study, we propose a novel fusion model of convolutional neural networks and Transformers to enhance performance in hyperspectral image classification, namely the dual-branch multi-granularity convolutional cross-substitution Transformer (DMCCT). The proposed model adopts a dual-branch structure to separately extract spatial and spectral features, thereby mitigating mutual interference and information loss between spectral and spatial data during feature extraction. Moreover, a multi-granularity embedding module is introduced to facilitate multi-scale and multi-level local feature extraction for spatial and spectral information. In particular, the improved convolutional cross-substitution Transformer module effectively integrates convolution and Transformer, reducing the complexity of attention operations and enhancing the accuracy of hyperspectral image classification tasks. Subsequently, the proposed method is evaluated against existing approaches using three classical datasets, namely Pavia University, Kennedy Space Center, and Indian Pines. Experimental results demonstrate the efficacy of the proposed method, achieving significant classification results on these datasets with overall classification accuracies of 98.57%, 97.96%, and 96.59%, respectively. These results establish the superiority of the proposed method in the context of hyperspectral image classification under similar experimental conditions.

Hyperspectral image (HSI) classification remains challenging due to the high dimensionality of spectral data, the strong coupling between spatial and spectral information, and the difficulty of learning robust representations under limited annotated samples. To address this, we propose 3D W-Net, a novel deep learning framework that integrates 3D convolution into the U-Net architecture and introduces a parallel–serial multi-scale feature learning strategy tailored for HSIs. The encoder incorporates a parallel spectraldimension multi-scale feature processing module, which employs three 3D convolution kernels with different spectral receptive fields to capture diverse spectral-scale features. An adjacent-scale selective fusion strategy is then applied to enrich spectral representations while reducing redundancy. The decoder features a serial spatial-dimension multiscale feature restoration module that progressively merges high-level semantic and lowlevel spatial details, preserving spatial–spectral correlations and mitigating information loss typically caused by flattening operations. This design forms a distinctive “W”-shaped feature propagation path that enhances multi-scale feature interaction. Extensive experiments on five public datasets(Botswana, Pavia University, Chikusei, Houston 2013, and WHU-Hi-HongHu) demonstrate that 3D W-Net achieves superior performance compared to six state-of-the-art methods, attaining overall accuracies of 99.6%, 99.1%, 100%, 99.3%, and 99.5%, respectively. The results highlight the effectiveness of the proposed parallel–serial multi-scale strategy in improving classification accuracy and generalization capability for complex HSI scenarios.

Graph-Transformer with spatial-spectral features fusion for hyperspectral image classification

Hyperspectral image classification is a critical task in remote sensing, but existing methods often employ fixed feature fusion strategies, making it difficult to adapt to the data characteristics of different scenarios. Additionally, there is a lack of effective synergy between multi-scale feature extraction and attention mechanisms. To address this issue, this paper proposes a dynamic gated fusion network with hierarchical multi-scale attention (DGFNet). This method comprises three core modules: the multi-scale feature aggregator (MSFA), which uses a pyramid expansion convolution structure to concurrently extract spatial features with different receptive fields, achieving comprehensive scale coverage from local texture to global context; the enhanced channel-spatial attention (ECSA) module, which employs multi-pooling strategies and a cascaded structure to achieve deep interaction between channel and spatial attention, thereby adaptively enhancing discriminative features; and the dynamic gated fusion module, which learns input-related fusion weights to adaptively adjust the contribution ratios of multi-scale features and attention features based on data characteristics. Experimental results on four benchmark datasets—Pavia University, Houston, Indian Pines, and WHU-HongHu—show that DGFNet achieves overall accuracy rates of 96.91, 97.12, 94.05, and 94.46%, respectively, representing significant improvements over existing state-of-the-art methods. Ablation experiments thoroughly validate the effectiveness and necessity of each module. Additionally, this paper systematically compares five different fusion strategies (cross-attention, hierarchical fusion, parallel fusion, recurrent fusion, and sequential fusion). Experimental results demonstrate that dynamic gated fusion outperforms other fusion methods in terms of classification accuracy, computational efficiency, and model stability. This method provides an efficient, accurate, and robust solution for hyperspectral image classification. The code will be published on https://github.com/willianbilledu-alt/DGFNet.

Hyperspectral image (HSI) classification has been extensively applied for analyzing remotely sensed images. HSI data consist of multiple bands that provide abundant spatial information. Convolutional neural networks (CNNs) have emerged as powerful deep learning methods for processing visual data. In recent work, CNNs have shown impressive results in HSI classification. In this paper, we propose a hierarchical neural network architecture called feature extraction with hybrid spectral CNN (FE-HybridSN) to extract superior spectral–spatial features. FE-HybridSN effectively captures more spectral–spatial information while reducing computational complexity. Considering the prevalent issue of class imbalance in experimental datasets (IP, UP, SV) and real-world hyperspectral datasets, we apply the focal loss to mitigate these problems. The focal loss reconstructs the loss function and facilitates effective achievement of the aforementioned goals. We propose a framework (FEHN-FL) that combines FE-HybridSN and the focal loss for HSI classification and then conduct extensive HSI classification experiments using three remote sensing datasets: Indian Pines (IP), University of Pavia (UP), and Salinas Scene (SV). Using cross-entropy loss as a baseline, we assess the hyperspectral classification performance of various backbone networks and examine the influence of different spatial sizes on classification accuracy. After incorporating focal loss as our loss function, we not only compare the classification performance of the FE-HybridSN backbone network under different loss functions but also evaluate their convergence rates during training. The proposed classification framework demonstrates satisfactory performance compared to state-of-the-art end-to-end deep-learning-based methods, such as 2D-CNN, 3D-CNN, etc.

Convolutional neural networks (CNNs), a kind of feedforward neural network with a deep structure, are one of the representative methods in hyperspectral image (HSI) classification. However, redundant information and interclass interference are common and challenging problems in HSI classification. In addition, if the spectral and spatial information is not properly extracted and analyzed, it will affect the classification performance of the network to a great extent. Aiming at these issues, this article proposes an HSI classification method based on an adaptive hash attention mechanism and a lower triangular network (AHA-LT). First, the attention mechanism is introduced in the preprocessing stage, which is composed of the spectral attention module and the adaptive hash spatial attention module in series. Then, the data processed by the attention mechanism are introduced into the lower triangular network (LTNet) to obtain the fused high-dimensional semantic features. Finally, we compress the features and obtain the output classification results through several fully connected layers. Among them, LTNet is composed of 2-D–3-D CNN and multiscale features. The network integrates the characteristics of multibranch, feature fusion, feature compression, and skip connections. Extensive experiments on four widely used HSI data sets show that the proposed method can obtain a great improvement in performance compared with the existing methods.

The growth of Hyperspectral Image (HSI) analysis is due to technology advancements that enable cameras to collect hundreds of continuous spectral information of each pixel in an image. HSI classification is challenging due to the large number of redundant spectral bands, limited training samples and non-linear relationship between the collected spatial position and the spectral bands. Our survey highlights recent research in HSI classification using traditional Machine Learning techniques like kernel-based learning, Support Vector Machines, Dimension Reduction and Transform-based techniques. Our study also digs into Deep Learning (DL) techniques that involve the usage of Autoencoders, 1D, 2D and 3D-Convolutional Neural Networks to classify HSI. From the comparison, it is observed that DL-based classification techniques outperform ML-based techniques. It has also been observed that spectral-spatial HSI classification outperforms pixel-by-pixel classification because it incorporates spectral signatures and spatial domain information. The performance of ML and DL-based classification techniques has been reviewed on commonly used land cover datasets like Indian Pines, Salinas valley and Pavia University.