Attention-based multi-scale network for hyperspectral image classification

Adaptive spectral-spatial feature fusion network for hyperspectral image classification using limited training samples

Traditional hyperspectral image (HSI) classification methods mainly include machine learning and convolutional neural network. However, they extremely depend on the large training samples. To obtain high accuracy on limited training samples, we propose a novel end-to-end spectral-spatial multi-scale network (SSMSN) for HSI classification. The SSMSN uses the multi-scale spectral module and the multi-scale spatial module to extract discriminative multi-scale spectral and multi-scale spatial features separately. In the multi-scale spectral module and spatial module, the multi-scale Res2Net block structure can learn multi-scale features at a granular level and increase the range of receptive fields by constructing the hierarchical residual-like connection within one single residual block. To alleviate the overfitting problem and further improve the classification accuracy on limited training samples, we adopt a simple but effective hinge cross-entropy loss function to train the SSMSN at the dynamic learning rate. A large number of experimental results demonstrate that on the Indiana Pines, University of Pavia, Kennedy Space Center, and Salinas Scene data sets, the proposed SSMSN achieves higher classification accuracy than state-of-the-art methods on limited training samples. Meanwhile, our SSMSN obtains less training and testing time than the popular AUSSC method.

Recently, deep learning methods, for example, convolutional neural networks (CNNs), have achieved high performance in hyperspectral image (HSI) classification. The limited training samples of HSI images make it hard to use deep learning methods with many layers and a large number of convolutional kernels as in large scale imagery tasks, and CNN-based methods usually need long training time. In this paper, we present a wide sliding window and subsampling network (WSWS Net) for HSI classification. It is based on layers of transform kernels with sliding windows and subsampling (WSWS). It can be extended in the wide direction to learn both spatial and spectral features more efficiently. The learned features are subsampled to reduce computational loads and to reduce memorization. Thus, layers of WSWS can learn higher level spatial and spectral features efficiently, and the proposed network can be trained easily by only computing linear weights with least squares. The experimental results show that the WSWS Net achieves excellent performance with different hyperspectral remotes sensing datasets compared with other shallow and deep learning methods. The effects of ratio of training samples, the sizes of image patches, and the visualization of features in WSWS layers are presented.

In hyperspectral image (HSI) classification task, effectively deriving and incorporating spatial information into spectral features is one of a key focus as it can largely influence the performance. Markov random fields (MRFs) are generative and flexible image texture models, and capable of effectively extracting spatial neighbourhood information along multiple spectral wavebands in an unsupervised way. Its parameter estimation process also shares strong compatibility with deep architecture, especially the convolutional neural networks. In this work, we propose an MRF based spectral-spatial fusion network (SSFNet) for HSI classification. Spatial features are extracted using MRF models and further fused with spectral information. Then the proposed SSFNet takes the fused features as input and produces reliable classification results. Comprehensive experiments conducted on the Indian pines and the Pavia university datasets are reported to verify the proposed method.

For hyperspectral image (HSI) classification, two branch networks generally use the convolution neural networks (CNNs) to extract the spatial features and the long short-term memory (LSTM) to learn the spectral features. However, CNN with a local kernel neglects the global properties of the whole HSI. LSTM doesn't consider the macroscopic and detailed information of spectra. In this paper, we propose a dual-view spectral and global spatial feature fusion network (DSGSF) to extract the spatial-spectral features for HSI classification, including a spatial subnetwork and a spectral subnetwork. In the spatial subnetwork, we propose a global spatial feature representation model based on the encoder-decoder structure with channel attention and spatial attention to learn the global spatial features. In the spectral subnetwork, we design a dual-view spectral feature aggregation model with view attention to learn the diversity of spectral features. By fusing the two subnetworks, we construct DSGSF to extract the spatial-spectral features of HSI with strong discriminating performance. Experimental results on three public datasets illustrate that the proposed method can achieve competitive results compared with the state-of-the-art methods. Code: https://github.com/RZWang-WH/DSGSF.

Hyperspectral image (HSI) classification accuracy has been greatly improved by employing deep learning. The current research mainly focuses on how to build a deep network to improve the accuracy. However, these networks tend to be more complex and have more parameters, which makes the model difficult to train and easy to overfit. Therefore, we present a lightweight deep convolutional neural network (CNN) model called S2FEF-CNN. In this model, three S2FEF blocks are used for the joint spectral–spatial features extraction. Each S2FEF block uses 1D spectral convolution to extract spectral features and 2D spatial convolution to extract spatial features, respectively, and then fuses spectral and spatial features by multiplication. Instead of using the full connected layer, two pooling layers follow three blocks for dimension reduction, which further reduces the training parameters. We compared our method with some state-of-the-art HSI classification methods based on deep network on three commonly used hyperspectral datasets. The results show that our network can achieve a comparable classification accuracy with significantly reduced parameters compared to the above deep networks, which reflects its potential advantages in HSI classification.

Recently, graph convolutional network (GCN) has drawn increasing attention in hyperspectral image (HSI) classification, as it can process arbitrary non-Euclidean data. However, dynamic GCN that refines the graph heavily relies on the graph embedding in the previous layer, which will result in performance degradation when the embedding contains noise. In this letter, we propose a novel dual residual graph convolutional network (DRGCN) for HSI classification that integrates two adjacency matrices of dual GCN. In detail, one GCN applies a soft adjacency matrix to extract spatial features, the other utilizes the dynamic adjacency matrix to extract global context-aware features. Subsequently, the features extracted by dual GCN are fused to make full use of the complementary and correlated information among two graph representations. Moreover, we introduce residual learning to optimize graph convolutional layers during the training process, to alleviate the over-smoothing problem. The advantage of dual GCN is that it can extract robust and discriminative features from HSI. Extensive experiments on four HSI data sets, including Indian Pines, Pavia University, Salinas, and Houston University, demonstrate the effectiveness and superiority of our proposed DRGCN, even with small-sized training data.

Although most deep learning-based methods have achieved excellent performance for hyperspectral image (HSI) classification, they are often limited by complex networks and require massive training samples in practical applications. Therefore, designing an efficient, lightweight model to obtain better classification results under small samples situations remains a challenging task. To alleviate this problem, a novel, lightweight parallel octave convolutional neural network (LPOCNN) for HSI classification is proposed in this paper. First, the HSI data is preprocessed to construct two three-dimensional (3D) patch cubes with different spatial and spectral scales for each central pixel, removing redundancy and focusing on extracting spatial features and spectral features, respectively. Next, two non- deep parallel branches are created for the two inputs, which design octave convolution rather than classical 3D convolution to facilitate light weighting of the model. Then two-dimensional convolutional neural network is used to extract deeper spectral-spatial features when fusing spectral-spatial features from different parallel layers. Moreover, the spectral-spatial attention is designed to promote the classification performance even further by adaptively adjusting the weights of different spectral-spatial features according to their contribution to classification. Experiments show that our suggested LPOCNN acquires a significant advantage on classification performance over other competitive methods under small sample situations.

Fuzzy graph convolutional network for hyperspectral image classification

Dual selective fusion transformer network for hyperspectral image classification.

Recently, convolutional neural networks (CNNs) have demonstrated impressive capabilities in the representation and classification of hyperspectral remote sensing images. Traditional CNNs require massive data to sufficiently train the network. To tackle this problem, graph convolutional network (GCN) has been introduced for hyperspectral image classification. GCN methods usually construct the graph from either spectral or spatial domain, which has not adequately explored the information in the joint spectral–spatial domain. In this article, we propose a superpixel spectral–spatial feature fusion graph convolution network for hyperspectral image classification (S3FGCN). S3FGCN can comprehensively use information in spectral, spatial, and spectral–spatial domains with limited data. Moreover, to enhance the performance, we explore a shared weights’ GCN in the spectral–spatial domain. To further improve the efficiency, superpixels are used to construct the adjacency matrix. Finally, dynamic sampling is adopted to make the model focus more on difficult samples. In the experiments on four datasets, S3FGCN demonstrates better accuracy compared with the state-of-the-art hyperspectral image classification methods.

Convolutional neural networks (CNNs), a kind of feedforward neural network with a deep structure, are one of the representative methods in hyperspectral image (HSI) classification. However, redundant information and interclass interference are common and challenging problems in HSI classification. In addition, if the spectral and spatial information is not properly extracted and analyzed, it will affect the classification performance of the network to a great extent. Aiming at these issues, this article proposes an HSI classification method based on an adaptive hash attention mechanism and a lower triangular network (AHA-LT). First, the attention mechanism is introduced in the preprocessing stage, which is composed of the spectral attention module and the adaptive hash spatial attention module in series. Then, the data processed by the attention mechanism are introduced into the lower triangular network (LTNet) to obtain the fused high-dimensional semantic features. Finally, we compress the features and obtain the output classification results through several fully connected layers. Among them, LTNet is composed of 2-D–3-D CNN and multiscale features. The network integrates the characteristics of multibranch, feature fusion, feature compression, and skip connections. Extensive experiments on four widely used HSI data sets show that the proposed method can obtain a great improvement in performance compared with the existing methods.

Recently, hyperspectral image (HSI) classification methods based on deep learning have demonstrated excellent performance. However, these deep learning methods still face two major challenges. One is that they require a large number of labeled samples, and the other is that training parameters takes a lot of time. In this paper, we propose a dual-branch deep stochastic adaptive Fourier decomposition (SAFD) network (DSAFDNet) to alleviate the aforementioned two issues in HSI classification applications. SAFD is a newly developed signal processing tool with solid mathematical foundation. It can be used to find common filters (i.e. convolution kernels) of a set of random signals or multi-signals. Since the convolution kernels obtained by SAFD decomposition are complex numbers, few deep learning methods directly deal with such complex convolution kernels. To this end, we propose a dual-branch network to extract deep features from hyperspectral images using both real and imaginary parts of convolutional kernels. After deep feature extraction using DSAFDNet, we further investigate the classification performance of different classifiers on the extracted features. Experimental results show that the proposed method outperforms some HSI classification methods with similar principles. Moreover, compared with other state-of-the-art deep learning methods, the proposed method can achieve better classification performance.

ABSTRACTConvolutional neural network (CNN) for hyperspectral image classification can provide excellent performance when the number of labeled samples for training is sufficiently large. Unfortunately, a small number of labeled samples are available for training in hyperspectral images. In this letter, a novel semi-supervised convolutional neural network is proposed for the classification of hyperspectral image. The proposed network can automatically learn features from complex hyperspectral image data structures. Furthermore, skip connection parameters are added between the encoder layer and decoder layer in order to make the network suitable for semi-supervised learning. Semi-supervised method is adopted to solve the problem of limited labeled samples. Finally, the network is trained to simultaneously minimize the sum of supervised and unsupervised cost functions. The proposed network is conducted on a widely used hyperspectral image data. The experimental results demonstrate that the proposed approach provides competitive results to state-of-the-art methods.

Over the past few years, deep learning has been introduced to tackle hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN) based methods have progressed. However, due to the high dimensionality of HSI and equal treatment of all bands, the performances of CNN based methods are hampered. The labels of land-covers often differ between edge and the center pixels in pixel-centered spatial information. These edge pixels may weaken the discrimination of spatial features and reduce classification accuracy. Motivated by the attention mechanism of the human visual system, the spatial proximity feature selection with residual spatial–spectral attention network is proposed in this article. It contains a residual spatial attention module, a residual spectral attention module, and a spatial proximity feature selection module. The residual spatial attention module aims to select the crucial spatial information, which assigns weights to different features by measuring the similarity between the surrounding elements and their central ones. The residual spectral attention module is designed for spectral bands which are selected from raw input data by emphasizing the valuable bands and suppressing the valueless. According to the spatial distribution of features, the spatial proximity feature selection module is used to filter features effectively. Experiments on three public data sets demonstrate that the proposed network outperforms the state-of-the-art methods in comparison.